Ads in a chat are harder than ads next to a search box. In a post published October 21, 2026, OpenAI engineers Ritendra Datta and Karthik Gopalratnam explain how they built the system that picks ads for ChatGPT, and the answer is unusual: an LLM acts as the conductor, calling the ad system's own tools in a loop, in a service walled off from the one that writes ChatGPT's answers. This article explains the design, why OpenAI made each choice, what the separation does and does not guarantee, and what remains unanswered.
The post lands the same day OpenAI said ChatGPT Ads is expanding to 89 countries with over 100,000 advertisers. Earlier we examined the business side, including the $1 billion run-rate claim and custom audiences by email and phone. This one is about the machinery.
TL;DR: the design in six points
| Design choice | What OpenAI says |
|---|---|
| Stateful ad sessions | Information is remembered across turns of a conversation and across conversations |
| Orchestrator | A custom, post-trained frontier LLM makes tool calls for retrieval, ranking, relevance and auctions |
| Separation | Ad selection runs in its own service, separate models and code, alongside answer generation |
| No-ad is normal | Returning no ad is an ordinary, successful outcome |
| Delivery is not display | A selected ad shows only if it arrives in time and ChatGPT approves it |
| Self-improvement loop | Strong models judge results and train smaller ones; humans supply feedback on taste |
Why not just match keywords like a search engine?
Traditional ad systems react to discrete events: a query, a page view. ChatGPT conversations do not produce clean events. The post gives an example from the engineering team: a chat begins with a general question about running, and several turns later asks about shoes for training. The commercial meaning shifts as each constraint arrives, and context builds up over sessions. A system that acted on the first message might retrieve the wrong intent entirely.
OpenAI lists four risks of treating each message as a standalone request: misreading the goal, starting the ad flow before the real need is clear, showing ads that get in the way, and wasting cost by rerunning the flow every time the request changes. Its answer is a stateful session that follows and adapts to the developing exchange instead of relying on a single flattened query.
A ladder of result cards with a green card on top, illustrating ranking candidate ads in ChatGPT
What does the orchestrator actually do?
The orchestrator is a post-trained LLM that runs the whole ads flow as an agent. OpenAI says it works at a high level like this:
- Summarize and embed. It turns the conversation into brief text summaries, then into semantic embeddings. It can write several queries to preserve separate needs; the post's example splits a Yosemite trip chat into two queries, one for hiking shoes appropriate to the trip and one for a brand's new product lines.
- Retrieve. Multiple indices, each using approximate-nearest-neighbor search over embeddings, produce a deliberately broad pool of candidates. Different ad types get different indices; the product-feed index, for example, is aware of positive or negative reviews.
- Filter. Candidates are removed if the creative or campaign is inactive, if targeting excludes the locale, if policy restricts them, if the budget is spent, if pacing says wait, if the placement is incompatible, or if one advertiser would take too many slots.
- Score. Tools assess semantic relevance, predicted engagement and historical performance, and filter again for relevance and safety.
- Nominate and auction. The orchestrator picks plausible candidates, but OpenAI stresses it does not choose the winner or set the price. An auction tool applies admission thresholds, pacing, bid features, quality-adjusted ranking, pricing and final selection.
- Final checks. The orchestrator can block an ad that feels inappropriate in context, and a request that is not ready in time simply times out.
The orchestrator can repeat this as the conversation develops, re-running tools with fresher context and returning better ads if they arrive in time. If the user chooses personalization, it also considers available nonsensitive information from past conversations.
How is the wall between ads and answers built?
The central trust claim is architectural. The ads flow runs alongside answer generation in a separate service, consuming the conversation under its own eligibility, quality and delivery constraints. OpenAI says the services share neither a content-generation model nor code, ChatGPT does not depend on the ads system, and if ads serving is slow or down, the organic response continues. The frontend gets a bundle of the ad creative and a model-generated preamble, a short neutral line explaining how the ad relates to the conversation, kept separate from the advertiser's text so ChatGPT's voice stays neutral. If the bundle arrives before the response finishes, the ad shows below it, labeled as sponsored; if not, it is dropped for that turn.
What this guarantees is a clear engineering boundary: ad code is not in the path that generates the answer. What it cannot guarantee from the outside is every behavior of the system, such as how the conversation is summarized for the ad service or what is logged. OpenAI points to its published ads principles for the commitments: conversations stay private from advertisers, customer data is not sold, ads are labeled, and advertising does not influence answers. Those remain company statements. For a related privacy investigation, see our report on the cookie that tracked users across sites.
How does the system improve itself?
The post describes a self-improvement loop. Standard logging records granular outcomes, such as a turn that was not eligible, a session that started with no retrieval, retrieval that returned nothing, candidates that failed eligibility, or an ad that was delivered and later got an interaction. OpenAI says it uses its strongest, highest-reasoning models to judge the results and to train smaller, cheaper models to orchestrate and retrieve. It also gathers human feedback: clicks and hides, user surveys, and advertiser input through Ads Manager. The authors say people "shape taste," so human feedback steers direction even more than model feedback.
The team credits Codex as a force multiplier in building the stack, with guardrails for reliable and safe code shipping and data-infrastructure plugins to tighten the loop between insight and fix. That connects to our coverage of Codex in business settings, and the same pattern, an agent building the system that runs on agents, appears in other 2026 stories.
What does this mean for advertisers and users?
For advertisers, the design implies that matching depends on the conversation's meaning, not on keywords you bid on. Copy and feeds that describe what a product is for will be easier to retrieve than copy that repeats a phrase. The "no ad is a success" principle also suggests lower fill rates than in search, and that relevance thresholds, not just bids, decide who appears. OpenAI's own account of new formats points the way: it mentions letting users talk directly to businesses through ads.
For users, the notable details are the memory and personalization language. Sessions are stateful across turns and conversations, and personalization is something users can choose, with the orchestrator considering nonsensitive information from past conversations when they do. Plans matter too: OpenAI says ads appear only on the Free and Go plans. See our test of visual ads in ChatGPT for how placements look in practice, and the Notch founders joining the ads team for where the product may be heading.
What is not answered
- No performance numbers. The post gives no click-through, conversion or latency figures, and no measure of how often an ad is selected but not shown.
- What counts as nonsensitive. The categories excluded from personalization are not listed in the post.
- How the boundary is audited. No independent review of the separation is cited.
- How the orchestrator is evaluated for harm. The post mentions integrity models and policy checks but gives no error rates, for example how often an ad appears in a sensitive conversation.
- Cost. Running an LLM orchestrator loop per eligible turn has a compute cost; OpenAI says only that smaller models can do the orchestration after training.
Why it matters beyond ads
The ads stack is a production example of an agent with tools running a latency-bound, high-stakes workflow, with a separate system watching for failure. Developers building agents will recognize the pieces: tool calls instead of one giant prompt, a stateful session, hard timeouts so the agent cannot block the main product, and an explicit "do nothing" outcome. The design also shows the trade OpenAI is making: it wants advertising revenue without letting it reshape answers, and it is betting that architecture, not just policy, is the reassurance users need.
Related reading
- ChatGPT Ads $1 billion run rate and the 89-country expansion
- ChatGPT custom audiences by email and phone, claimed vs verified
- ChatGPT visual ads test
- Notch founders join OpenAI's ads team
- OpenAI's Advertise in ChatGPT launch
- The ChatGPT cookie that tracked users across sites
Details reflect OpenAI's October 21, 2026 engineering post; the system may change.