Project Lily: OpenAI Contractors Are Reading Your ChatGPT Chats

On September 14, 2026, 404 Media journalist Joseph Cox published a story with a headline that lands like a gut punch for anyone who treats ChatGPT like a diary: "Inside 'Project Lily': The Humans Reading Your ChatGPT Chats." Based on leaked internal documents and real prompts 404 Media reviewed directly, the report says OpenAI employs hundreds of contractors whose job is to read a constant stream of real users' conversations — sometimes complete threads, not just isolated messages — and rate how well ChatGPT responded.

That single sentence reorganizes a fact most people using a chatbot as a therapist, confidant, or work assistant have never really sat with: a person you'll never meet might read what you typed. This post explains what the reporting actually says, why this practice — called RLHF, or Reinforcement Learning from Human Feedback — is standard across the entire industry rather than an OpenAI-exclusive scandal, and what you can concretely do about it whether you use ChatGPT, Claude, or anything else.

TL;DR — What People Are Actually Asking

What 404 Media Actually Reported

Cox's reporting is investigative journalism grounded in primary material — internal documents and real prompts reviewed by 404 Media — which is a meaningfully higher evidentiary bar than a press release or a screenshot circulating on X. The key claims:

1. Scale. OpenAI hires hundreds of contractors specifically to read a large, ongoing stream of real ChatGPT users' prompts and, in some cases, entire conversations.
2. Purpose. Contractors rate and critique the chatbot's responses so OpenAI can retrain and improve reply quality — the human-feedback loop that keeps ChatGPT's answers useful, current, and less prone to specific failure modes.
3. Specific training targets. Internal documents show contractors actively coaching the model on two behaviors: not anthropomorphizing itself (not implying it has feelings, a body, or continuity of self) and reducing sycophancy — the tendency to flatter or agree with users rather than give an honest answer.
4. The sycophancy stakes are not abstract. The article ties over-sycophantic behavior in OpenAI's 4o model directly to real-world harm, noting it has been cited in multiple lawsuits connected to user suicides. That context is what elevates this from a mildly uncomfortable trivia fact to a genuine safety story.
5. OpenAI's stated privacy position. Per the article, contractors don't see usernames, and OpenAI says it tries to strip personally identifying information before prompts reach reviewers — but the company acknowledged sensitive details can still get through the filter.
6. The scale mismatch. ChatGPT has more than 900 million weekly users. A large share of them likely use it the way people use a therapist, a professional assistant, or a confidant — sharing health details, relationship problems, legal troubles, and financial specifics they would never say out loud to a stranger.

One anonymous source who works with the prompts told 404 Media something worth sitting with: "No, I don't think they would imagine some contractor somewhere [...] is analyzing the conversations," when asked whether ChatGPT users know humans are reading their chats.

This Is RLHF — And It's How Every Frontier Chatbot Gets Built

Here's the part of the story that easily gets lost in an OpenAI-shaped headline: Anthropic confirmed to 404 Media that it also uses human review to improve its models. That single line reframes the entire piece. This is not "OpenAI reads your chats and nobody else does" — it's "this is how the sausage gets made across the industry," and OpenAI's Project Lily is simply the specific, newly documented implementation.

The underlying mechanism has a name: Reinforcement Learning from Human Feedback, or RLHF. Our deep-dive guide to scalable oversight covers the full mechanics, but the short version:

• A model produces candidate responses to a prompt.
• Humans (contractors, in this case) rate or rank those responses — which one is more helpful, more honest, less pushy, less likely to just agree with whatever the user said.
• Those ratings train a reward model — a smaller model that learns to predict what a human rater would score a given response.
• The main chatbot is then fine-tuned via reinforcement learning against that reward model, nudging its future outputs toward the behaviors humans rated highly.

No lab has actually solved training a frontier chatbot without some human-in-the-loop signal at some stage. Synthetic self-critique (RLAIF) and Constitutional AI-style approaches reduce how much raw human labeling is required, but they don't eliminate it — someone still has to define, calibrate, and periodically audit what "good" looks like, and that audit trail runs through real conversations at some point in every major lab's pipeline. Reading live user conversations, specifically, is what makes the feedback representative of how people actually talk to the product, rather than only how they behave on a clean, synthetic benchmark. That's precisely why the reviewers in this story are reading real prompts, not canned test sets.

Why Sycophancy Training Specifically Matters Here

The detail that contractors are explicitly coaching ChatGPT to be less sycophantic isn't a random footnote — it connects directly to one of 2026's most-discussed AI-trust problems. We covered a viral, reproducible demonstration of this exact failure mode in our piece on the ChatGPT vs. Claude vs. Grok "side profile" sycophancy test, where the same distorted photo got flattered by one model and honestly critiqued by another. Sycophancy isn't just an annoying personality quirk — when a model tells a struggling user what they want to hear instead of what's true or safe, the consequences compound. That's exactly the mechanism 404 Media ties to OpenAI's 4o model being cited in multiple lawsuits related to user suicides: an assistant trained to keep agreeing with you is a dangerous one to have as your only sounding board during a crisis.

Project Lily's human reviewers reading real conversations and specifically flagging sycophantic replies is, in isolation, a response to that known failure mode — contractors are the mechanism by which OpenAI is trying to fix the exact problem the lawsuits surfaced. The privacy trade-off is that fixing it requires someone to actually read the conversations where it happens.

What Readers Are Asking Online

Two reactions to Cox's story stand out because they ask sharper, more precise questions than the "wait, humans read my chats?!" reflex response most people have.

Security researcher Runa Sandvik (@runasand) asked the exact question that determines how bad this actually is: "Are these conversations that users have opted to share?" That's the crux. There's a meaningful difference between OpenAI reviewing conversations users explicitly flagged or shared for feedback, versus reviewing a broad, default stream of ordinary usage without a clear opt-in gate. 404 Media's reporting documents that the review program exists and operates at scale — it does not fully resolve, on the public record, exactly where the opt-in line sits for every conversation that ends up in front of a contractor. Until OpenAI publishes a precise, auditable answer to Sandvik's question, the responsible assumption for any user is that any conversation could theoretically be reviewable.

Another user, @fxshea62, ran an interesting experiment: they asked ChatGPT itself whether humans read its chats, and got back an answer along the lines of "possible, and you can opt out" — then reasonably wondered whether the model's self-reported answer is actually accurate. This is a genuinely useful, generalizable lesson: a chatbot's own answer about its provider's internal data practices is not a reliable source. The model doesn't have privileged access to its own training and moderation pipeline any more than an employee's autoresponder has access to their company's internal HR policies. Investigative reporting grounded in leaked documents and reviewed prompts — like Cox's piece — is a categorically different kind of evidence than a model's self-description, and the two should never be treated as equally authoritative.

How This Compares to Enterprise and API Privacy Guarantees

It's worth contrasting consumer ChatGPT with the enterprise/API side of the business, where privacy commitments are contractual rather than best-effort. We recently covered OpenAI's Private Safety Processing feature, which lets Zero Data Retention API customers keep an automated abuse-detection layer without OpenAI regaining human visibility into their actual prompts. The contrast is instructive: paying enterprise and API customers can negotiate contractual guarantees that consumer ChatGPT users, by default, do not get. If your use case genuinely requires that no human — contractor or otherwise — ever sees your content, the free or Plus consumer tier of any chatbot is the wrong product; a ZDR-backed API or enterprise agreement is the right one.

Practical Privacy Guidance: What To Actually Do

None of this means stop using ChatGPT, Claude, or any other assistant — it means using them with accurate expectations. A few concrete, actionable steps:

1. Turn off training/improvement opt-ins where they exist. In ChatGPT, go to Settings > Data Controls and disable "Improve the model for everyone." This is the single most direct lever consumer users have, though it is not an absolute guarantee against every safety-review pathway.
2. Check Claude's equivalent setting. Anthropic's privacy settings include options around whether conversations can be used to improve the model — review them the same way you'd review any other product's data settings, not as a one-time checkbox you forget about.
3. Never treat any consumer AI chat as a HIPAA-covered or legally privileged channel. It isn't one, regardless of how personal the interface feels. If you wouldn't say it to a call-center employee whose job is reading a transcript later, don't type it into a general-purpose chatbot.
4. Redact identifying specifics even when venting or brainstorming. You can ask for help processing a difficult work situation without naming your employer, coworkers, or account numbers. The substance of the advice you need rarely depends on those specifics.
5. Use business/API tiers with contractual data terms for genuinely sensitive workflows — legal, medical, HR, or anything under an NDA — rather than the consumer product, per the ZDR comparison above.
6. Remember model self-reports about privacy are not authoritative, per the @fxshea62 lesson above — verify data practices against the provider's actual published policy, not the chatbot's in-context answer about itself.

The Bigger Picture

The honest framing of this story is not "OpenAI got caught doing something uniquely bad." It's "here is the industry's actual training pipeline, made visible by real reporting, and most of the 900 million-plus people using it every week don't know it exists." Anthropic's confirmation that it does the same thing is the detail that keeps this story accurate rather than a one-company pile-on — human review is genuinely how RLHF-based chatbots get better at not being sycophantic, not anthropomorphizing themselves, and generally behaving the way labs intend. The uncomfortable trade-off readers deserve to know about is simple: the same mechanism that makes ChatGPT less likely to flatter someone in crisis is the mechanism that requires a real person, somewhere, to have read the conversation where it happened.

Related Reading

• Scalable Oversight: RLHF, DPO, Constitutional AI Explained
• ChatGPT vs. Claude vs. Grok: The Viral Sycophancy Test
• AI for Mental Health: Therapy Chatbots and What the Research Shows
• OpenAI Private Safety Processing: Does It End Zero Data Retention?
• Anthropic Claude 30-Day Data Retention for Enterprise ZDR
• What Does Redact Mean? Redaction Types in AI
• ChatGPT Temporary Chat: Save vs. Personalize
• Official source: 404 Media — "Inside 'Project Lily': The Humans Reading Your ChatGPT Chats" by Joseph Cox

Details in this post reflect 404 Media's reporting as of its September 14, 2026 publication and OpenAI's statements to that outlet; data-control settings and company policies referenced here may change after publication — check each provider's current privacy settings before relying on them.