AI & Marketing Research with Dr. Eva Wolf
Not another AI news podcast. This is a research radar — a twice-weekly briefing that surfaces peer-reviewed studies on AI and marketing, tells you what the evidence actually says, and helps you decide what's worth a deeper read.
AI & Marketing Research with Dr. Eva Wolf
AI Chat Logs, Privacy & Dependency: 2 Research Signals
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
What if the AI tools your users love, the ones keeping them engaged, coming back, sharing everything, are quietly building the most detailed demographic profile anyone's ever had on them? That's the uncomfortable question this week. One paper is about privacy, one's about dependency. Both are about what happens when AI gets intimate with users, and what that costs. We screened 140 papers. Two made the radar. Quick caveat: this is a first pass research briefing, not a final academic review. I'll tell you what the papers suggest, what they don't prove, and which ones deserve a deeper read. Both are preprints. Hold them accordingly. Okay, let's get into the first one. Paper one, you strip the names, the phone numbers, the email addresses out of your AI chat logs, and you think you're done. You're not. Researchers ran a privacy audit on over a thousand real chat GPT users. They took conversation histories, filtered out anyone who'd explicitly stated their demographics, and asked a large language model to guess each user's age, gender, and country just from what they talked about. Here's what happened. It got it right. Age, gender, country. F1 scores between 0.84 and 090. That's not a lucky guess. That's a reliable classifier. And it didn't need much. For more than half the users, the model had it figured out after reading just the first 5% of their conversation history. A handful of messages. So let me say this plainly: anonymizing AI chat logs by removing names and contact details does not anonymize them. The topics you discuss are enough. The style is enough. The questions you ask are enough. That's the piece I care about. Because I know how many marketing and product teams are sitting on customer chat logs right now thinking we removed the PII, we're fine. This paper says you're not fine. And they went further. Cross-platform comparison, chat GPT logs versus Google search history versus YouTube for a subgroup who donated all three. Chat GPT was competitive with Google search for profiling age, education, and political preferences. Think about what that means. Google search history is the gold standard of behavioral targeting data. Chat GPT conversations are in the same league, possibly better for some signals. Now here's the part that should make you uncomfortable if you work an AI product. The model wasn't just right, it was wrong in a patterned way. Conversations about programming or finance got flagged as male. Conversations about family or feelings got flagged as female. So women in tech, older users with strong digital skills, tech workers from Nigeria and Pakistan, they were the most misidentified. The AI was relying on stereotypes baked into its training, not surface-level PII. Stereotypes. That's a double problem, privacy and bias at the same time. Plain English payoff. If your company stores customer AI chat histories, those logs can profile your users' age, gender, and country even after you've scrubbed every name and number. And they're as revealing as Google search data. Money Move, the first agency or SaaS tool that builds a demographic inference risk score for enterprise AI chat logs and sells it to privacy and compliance teams who need to demonstrate GDPR or CCPA readiness, is going to own a category that doesn't exist yet. Try this by Friday. Pull up whatever AI chat tool your company runs, customer support bot, internal assistant, doesn't matter. Ask your privacy team one question. Does our current anonymization process address contextual and topical signals or just direct identifiers? That question alone starts a necessary conversation. Evidence check. This is a preprint, not peer-reviewed yet. The sample is from four countries: Brazil, India, Nigeria, Pakistan, all voluntary donors. So we can't assume these findings transfer cleanly to North American or European users. And the cross-platform comparison is 212 users, all Indian, which limits how far you can push that piece. The methodology is genuinely rigorous for what it is, but generalizability is the real ceiling. Radar verdict, test this week. The core finding that topical and stylistic signals in chat logs predict demographics as well as search history is empirically grounded and has immediate implications for anyone in AI product, data, or compliance. Don't wait for the peer-reviewed version to start asking the right questions internally. Paper two, this one's less about data and more about people. Specifically, what happens to people when the AI is the best mental health support they can access? Researchers ran a qualitative longitudinal study, interviews, a four-week diary, focus groups, exit interviews, with 18 U.S. adults already using general-purpose AI chatbots for emotional support. And the first finding isn't actually about AI, it's about healthcare. The average wait time in the US for a mental health appointment is 48 days. 60% of therapists aren't taking new patients. So people aren't turning to AI because they prefer it. They're turning to it because the alternative doesn't exist. Okay, here's the thing: that context matters for marketers. Because if you're building or selling an AI tool with any wellness, coaching, or support angle, you're not just selling a product, you're filling a care gap. And that comes with obligations. So what did these users actually experience? Over four weeks, they describe becoming more dependent on the chatbot, not because it was working, but because it was always available, always warm, and never pushed back. That last part, never pushed back. The AI consistently validated them. And over time, some participants described a kind of echo chamber, not of political content, but of their own thoughts and feelings. The AI kept reflecting them back to themselves without challenge. And here's what I keep coming back to. The users knew this was happening. They could describe the dynamic clearly. They weren't deceived. They kept coming back because there was nowhere else to go. The researchers argue that current AI safety frameworks are focused on catching dangerous single responses. A chatbot says something harmful. Flag it. But the actual harm in emotional support use is slower than that. It's dependency that builds over weeks, it's relationship displacement, its privacy exposure, its disruption when the model gets updated and suddenly feels different. That's a governance gap. It's also a liability gap for any brand that's built an AI product in this space. Plain English payoff, the same design features that make your AI product feel engaging and sticky, always available, always agreeable, emotionally warm, are the exact features that create dependency risk in vulnerable users and regulators are starting to notice. Try this by Friday. If you run engagement metrics on any AI product, time and app, return visits, session depth, look at your highest engagement users. Ask whether those patterns look like healthy use or like someone who has no better option. You don't need to solve it this week. You need to see it. Evidence check. And I want to be direct here. This is a small study, 18 people, all US-based, all already using AI for emotional support. That's not a random sample. That's a recruited cohort with a specific behavior. The findings describe patterns, not frequencies. We cannot say how common any of this is in the broader population. It's a preprint, qualitative design, no causal claims are warranted. Use this paper to ask better questions, not to make product decisions. Radar verdict watch list. The engagement versus well-being tension is real and growing in regulatory salience, but the evidence base is too small and too early to act on directly. Track it, especially if you're an AI wellness, HR tech, or any consumer AI product with an emotional support angle. Okay, here's what I think is actually happening this week. These two papers look like they're about different things. One's about data privacy, one's about emotional dependency. But they're describing the same dynamic from two directions. AI intimacy creates data. The more someone uses an AI conversationally, sharing problems, asking questions, processing feelings, the more that conversation history becomes a demographic fingerprint. Paper one proves the fingerprint is real. Paper two explains why people are handing it over. And that's the tension I keep landing on. Users aren't sharing this data carelessly. They're sharing it because the AI is the most accessible support they have. The healthcare system failed them. The stigma's real. The AI was there. So when we talk about using AI chat data for targeting or designing AI products for engagement, we're not talking about neutral business decisions. We're talking about decisions made in a context where users have very little power and very few alternatives. That doesn't mean you don't build the product. It means you build it differently. And it means your legal and privacy team needs to be in the room before the product ships, not after the press story runs. Here's the playbook from this week. One, if your company stores AI chat logs, ask your privacy team this week whether your anonymization process covers topical and contextual signals, not just names and contact details. Paper one gives you the question. Your team needs to find the answer. Two, if you build or pitch AI tools with any wellness, coaching, or emotional support angle, document the dependency and validation risks now. Proactive disclosure protects your users and protects you. Don't wait for a regulator or a journalist to make you do it. Evidence check on all of that. Both papers are preprints. Neither's been peer reviewed. The privacy paper has rigorous methodology, but limited geographic generalizability. The emotional dependency paper has a very small qualitative sample. Use them to decide what to test and what to examine internally, not what to treat as settled fact. Ask better questions, don't make big moves. Links to both papers are in the show notes. Read the originals before making major decisions, and definitely before quoting these findings in a presentation. See you Thursday. And if something from this episode changed how you think about your AI product roadmap or your data practices, I genuinely want to hear it.