Ask an LLM to classify a search query, a support ticket, or a product listing into one of a few hundred categories, and the textbook answer is structured outputs: send the model the entire legal taxonomy as a Literal type or JSON schema enum, and force it to pick one. It works — until you count the tokens. A Hacker News post from Doug Turnbull (softwaredoug.com) that hit 216 points and 85 comments argues for the opposite move: don't make the LLM classify at all. Let it hallucinate, then resolve the hallucination with embeddings.
The technique is a direct descendant of HyDE, a well-known retrieval-augmented generation trick, applied to a problem — classification into a large closed vocabulary — that HyDE wasn't originally built for. Commenters on the thread spotted the connection immediately, and Turnbull confirmed it was exactly the intent.
TL;DR
| Question | Answer |
|---|---|
| What's the standard approach? | Ship the full taxonomy to the LLM as a structured output schema (Pydantic Literal, JSON schema enum) and force a pick |
| What's the alternative? | Ask a cheap/small model to freely invent a plausible category it was never shown, then embed that fake answer and nearest-neighbor match it to real categories |
| Why is it cheaper? | The taxonomy never goes to the LLM — only a pre-computed, in-memory embedding index sees it, so you can use much smaller models at high volume |
| Where does it come from? | It's a variant of HyDE (Hypothetical Document Embeddings), originally a search technique, not a classification one |
| What's the catch? | Works best on common-knowledge domains the model already has priors about; degrades on niche or private taxonomies |
| Is there a hybrid version? | Yes — search first for a shortlist of likely categories, then classify only within that shortlist |
The problem: taxonomies are expensive to ship every call
Turnbull's example uses the Wayfair WANDS dataset, a product-search relevance benchmark with a deep e-commerce category tree — paths like Furniture / Living Room Furniture / Coffee Tables & End Tables / Coffee Tables. Hundreds of these leaf categories exist.
The conventional way to classify a query like "brown coffee table" into one of them is structured output: define every category as a Literal string or a JSON schema enum, put the whole list in the prompt or schema, and let the model pick. This is reliable — the model literally cannot return an invalid category — but it means every single classification call pays the token cost of the entire taxonomy, whether the query needed to see all 500 options or not. At volume, and with a taxonomy that keeps growing, that cost compounds, and it pushes you toward models large enough to reliably hold a big enum in context rather than a small, cheap one.
The alternative: hallucinate a fake category, then resolve it
Turnbull's fix skips the taxonomy entirely on the classification call. Instead, you prompt a cheap model — the post uses gpt-5.4-mini as the example — with something like:
Create a novel, never-seen-before furniture/home-goods
classification that best fits this search query.
Query: "brown coffee table"
The model has never seen the real taxonomy, so it invents something plausible-sounding but not necessarily verbatim-correct — say, Furniture / Living Room / Tables / Coffee. That string doesn't exist anywhere in the real WANDS taxonomy. But it's semantically close to the real leaf category, Furniture / Living Room Furniture / Coffee Tables & End Tables / Coffee Tables.
That closeness is the whole trick. You take the hallucinated string, embed it with a cheap local model — Turnbull uses MiniLM — and run a nearest-neighbor / dot-product similarity search against embeddings of every real category, computed once, ahead of time, and held in memory. The hallucinated embedding lands close enough to the real category's embedding to resolve correctly, even though the LLM that generated it never saw the real option.
# 1. Precompute once, in memory
real_categories = [...] # e.g. WANDS taxonomy leaf paths
category_embeddings = embed_model.encode(real_categories) # MiniLM or similar
# 2. Per query — cheap model, no taxonomy in the prompt
hallucinated = cheap_llm.generate(
f'Create a novel, never-seen-before furniture/home-goods '
f'classification that best fits this search query.\n\nQuery: "{query}"'
)
# 3. Resolve the hallucination to a real category
query_vec = embed_model.encode([hallucinated])
best_match = nearest_neighbor(query_vec, category_embeddings)
Turnbull reports this being "extremely helpful," and describes it landing close to — sometimes only slightly behind — the accuracy of shipping the full vocabulary to a larger closed-model API, at a fraction of the cost, when tested against a nano-tier model. Those specific accuracy and cost figures are his own reported numbers from his testing, not independently verified benchmarks.
Why this is HyDE, not a new trick
HyDE — Hypothetical Document Embeddings, from the 2022 paper on arXiv — is a retrieval technique for search, not classification. Instead of embedding a user's literal query and comparing it to documents, HyDE has an LLM generate a hypothetical answer first, embeds that hypothetical answer, and compares it to real document embeddings. The insight: a terse query and its matching document often sit far apart in embedding space, but a full, plausible hypothetical answer — even a wrong one — tends to land much closer to the real matching document, because it's written in the same register and level of detail as the thing it needs to match.
Multiple commenters on the Hacker News thread flagged this immediately, and Turnbull confirmed it directly: "the embedding of the query is not necessarily close to the embedding of the answer... if you generate a hallucinated answer, it can line up with the actual document better." Swap "document" for "category label," and the search technique becomes a classification technique. The category taxonomy is the "document" being retrieved, and the hallucinated classification string is the "hypothetical document" doing the retrieving.
"Why not just embed the query directly?"
The most common pushback in the thread: why hallucinate anything? Why not embed the raw query — "brown coffee table" — directly and run nearest-neighbor against the category embeddings, skipping the LLM call entirely?
Turnbull's response was that restating the query "in the language of the taxonomy" measurably improves retrieval accuracy versus embedding the raw query as-is. A raw query like "blue shoes" mixes an attribute (color) with an item type in a way that's ambiguous for an embedding model to disentangle; a restated hypothetical category path forces that disentanglement before embedding happens. The hallucination step isn't decoration — it's doing real work reformatting an ambiguous query into a structurally comparable form.
No consensus was reached on the thread about whether this beats embedding the raw query with a strong, modern bi-encoder tuned to already "support queries" directly — several commenters suggested it's genuinely domain-dependent and worth A/B testing both approaches rather than assuming one wins universally.
The important caveat: this needs common-knowledge priors
A caveat came up repeatedly in the discussion, and it's the part worth taking most seriously before adopting this pattern: hallucinate-then-resolve works when the LLM already has reasonable priors about the domain.
For something like e-commerce furniture categories, a small model has seen enough training text about furniture, home goods, and retail taxonomies to invent something in the right neighborhood. But for a niche, private, or highly domain-specific taxonomy — internal support-ticket codes, a proprietary product line, jargon unique to one company — the model has no training signal to hallucinate a plausible answer from. In that case, the hallucinated embedding can drift farther from the real answer, not closer, because the model is generating noise rather than an informed guess.
This is the same limitation that shows up whenever you lean on a model's parametric knowledge instead of grounding it — see the broader trade-offs in RAG vs. fine-tuning and why models hallucinate in the first place. If your taxonomy is common-knowledge (product categories, content genres, general support-ticket types), this technique has real room to work. If it's proprietary or niche, expect it to underperform sending the model the real options.
The recommended enhancement: shortlist first, then classify
Turnbull's own suggested improvement blends both approaches rather than picking one: run cheap BM25 or embedding search first to pull a shortlist of the most likely real categories for a given query, aggregate them, and then have the LLM classify only within that narrowed shortlist — instead of either the full taxonomy or a completely free hallucination. This keeps prompt size small (a shortlist of 5-10 candidates instead of hundreds) while still giving the model real options to ground its answer in, rather than asking it to invent one from nothing.
Where else this pattern shows up
Commenters connected the "generate, then resolve via embedding similarity" pattern to a few adjacent problems:
- Semantic routing. semantic-router is an existing open-source project doing something conceptually related — routing requests to the right handler using semantic similarity rather than a hardcoded classifier or keyword match. It's a useful reference for anyone building this pattern from scratch inside an agent harness.
- Ad-hoc clustering. Applying the same generate-then-resolve idea to cluster support tickets or user complaints into categories nobody predefined: embed all the records, cluster the embeddings, sample each cluster, and ask the LLM to name or classify the cluster after the fact rather than before.
- Entity resolution. Resolving arbitrary extracted values — brand names pulled from free text, for instance — to a canonical, closed list using the same hallucinate-and-embed resolution step, instead of a rigid extraction schema.
Where this fits next to structured outputs and RAG
This technique isn't a replacement for structured outputs generally — it's a cost optimization for one specific shape of problem: mapping free text into a large, fixed, mostly-static vocabulary where the LLM already has reasonable domain priors. For dynamic outputs, tool calls, or anything where correctness (not just plausibility) matters more than cost, standard structured output with JSON schema or tool-use schemas is still the right call.
It's also a useful reminder that RAG and classification are closer to the same problem than they look. Both are fundamentally about matching a query representation to the right item in a fixed corpus — HyDE just proved that the "right" query representation is sometimes a hallucinated answer, not the literal query.
Related reading
- What is HyDE? — dictionary entry
- What are embeddings and vector search? Complete guide
- Semantic vs. vector vs. hybrid search guide
- RAG vs. agentic RAG
- Grounding: RAG vs. fine-tuning decision guide
- Structured output and JSON mode prompting guide
- Structured output vs. tool use / JSON schema guide
- Why AI models hallucinate — and how to catch it
- Top 10 open and closed source embedding models 2026
- Agent harness: DAG planning, tiered memory, budget pressure
External: Doug Turnbull's original post (softwaredoug.com), HyDE paper on arXiv, Wayfair WANDS dataset, semantic-router on GitHub
Model names, benchmark figures, and specific claims are accurate as of the publication date and attributed to Doug Turnbull's original post and the Hacker News discussion around it; explainx.ai has not independently re-run these cost or accuracy comparisons.