Apple SupportNet and KeyNet: Learned Routing That Makes RAG Retrieval Up to 8x Faster

Apple has a way to make the retrieval step of RAG faster by learning from the questions users actually ask. In a post published November 5, 2026, Apple Machine Learning Research describes amortized maximum inner product search (MIPS): small neural networks, trained once offline for a fixed database, that route or rewrite queries so a vector index finds the right match faster (Apple ML Research). The paper, "Amortizing Maximum Inner Product Search with Learned Support Functions," was presented at ICML 2026.

All numbers below are Apple's own, measured on its setup. They have not been independently reproduced.

TL;DR: the questions people ask first

Why retrieval speed matters for RAG

In retrieval-augmented generation, a short user question becomes a query vector. The system searches millions of document vectors for the best match, then prepends that document to the prompt. Each query pays the search cost before the model starts to write.

The standard shortcut is clustering. Split the keys into k clusters, compare the query to each cluster's centroid, then search only the closest clusters. That gives roughly a factor-of-k speedup, but routing by centroid can miss. A cluster may hold a key far from its centroid yet close to the query, and centroid routing skips it.

Apple also points to a mismatch most indexes ignore. In RAG, queries are short user questions, while keys are long, curated passages, so the two look different. And production systems often serve a stable, well-characterized query stream, a signal generic methods leave unused. For background on how embeddings and search fit together, see our guide to embeddings and vector search and the semantic vs vector vs hybrid search comparison.

The key idea: MIPS is a support function

Apple notes that the function mapping a query to its largest inner product with any key in a set is the support function of that set, a standard object in convex analysis. Two properties follow:

1. It is convex, so a class of networks built for convex functions, input convex neural networks (ICNNs), can approximate it.
2. Its gradient at a query returns the best-matching key itself.

So you can treat retrieval as regression. Sample queries from the expected distribution, compute the true best match once by exhaustive search, store it, and train a network to reproduce it. That one-time cost is the "amortization."

Two models

SupportNet learns the support function of each cluster, using a loosely constrained ICNN. Instead of asking how close the query is to a centroid, it asks "How good is the best key inside this cluster for this query?" It replaces centroid routing in an inverted file (IVF) index.

KeyNet outputs the best-matching key directly for a query. A consistency loss derived from Euler's theorem for homogeneous functions keeps its predictions tied to the support function. In practice it works as a query rewriter: predict where the best match likely sits, then let an unchanged IVF index search there.

Apple says training separate networks for the two tasks worked better than one network doing both.

Results on Natural Questions

Apple trained model sizes XS, S, M and L, equal to 1%, 5%, 10% and 20% of the database size, on Natural Questions split into 128 clusters.

SupportNet routing.

• The small SupportNet hits 95% routing recall in 1.74 ms versus 3.36 ms for centroid routing, about 1.9x faster. The large model hits 97% in 1.74 ms.
• Apple says this timing includes the extra compute of running the networks.
• Scanning just 5% of the database, learned routers recover the true partition about 90% to 93% of the time, up to 92.6% for the mid-size model, versus 87.5% for the centroid baseline.

KeyNet with an IVF index (FAISS).

• Reaching 99% recall takes up to about 8x less time with the largest model: 0.23 ms versus 1.8 ms for the raw query.
• Scanning 0.001% of the database, KeyNet-mapped queries retrieve the true key 89% to 97% of the time, up to 96.7% for the largest model, versus 86% for the raw query.
• Reaching 99% recall takes up to about 11x fewer keys: 0.009% versus 0.10% for the raw query.
• Latency includes the cost of mapping the query. Apple says the preprocessing overhead is repaid by faster search downstream, at every model size.

Apple says it studied other datasets too, but the post only shows Natural Questions. FAISS is Meta's open-source similarity search library, and it is the IVF index used here. Apple also shows five example questions, such as "Who wrote The Eye of the World?", where the top-3 hits from the KeyNet-mapped query beat the raw query.

Reading the numbers without overreading them

Two details in Apple's post change how to read the speedups.

First, the two result sets measure different things. The SupportNet figures are about routing recall: how often the cluster holding the true best key is among the clusters the router chooses to probe. The KeyNet figures are about finding the true key itself through an IVF index. A 1.9x gain in routing is not the same claim as an 8x gain in end-to-end search, and the two should not be multiplied together.

Second, the latency comparisons include the networks' own compute. Apple says the SupportNet plot "explicitly factors in the additional compute incurred by running our small NNs," and that KeyNet latency is the total time to map a query and search, while the raw-query baseline counts only search. That is the fair way to compare, because a faster index that needs a slow preprocessing step gains nothing.

The other thing to watch is the baseline. Centroid routing is the common default in IVF systems, so the gains are against a realistic starting point. They are not against every modern index, such as graph-based indexes, which the post does not test. Before adopting the method, run your own index as the baseline.

Which one should you use?

Apple's guidance is practical:

Both integrate with existing approximate-search infrastructure and are, per Apple, "reasonably robust to mild distribution shift in the query patterns."

What this means for what you build

If you run RAG over a fixed corpus with a steady query pattern, such as a help center, a product catalog or an internal wiki, this idea targets your retrieval latency and recall without changing the index. The model is trained on sampled queries, so you need a representative query log or a way to synthesize one. If your corpus changes daily or your queries are unpredictable, the one-time training cost becomes recurring, and generic indexing is simpler.

Retrieval is also one of several places where RAG quality is won or lost. See our posts on RAG context injection pipeline design, RAG vs agentic RAG, and why agent memory systems underperform naive RAG. For embedding-side choices, see embeddings on device with EmbeddingGemma 2.

How to test the idea on your own data

Apple's post does not announce a code release, so treat this as a method to reproduce.

1. Build an IVF index on your key vectors and measure recall at a few cluster-probe budgets.
2. Log or synthesize a few thousand realistic queries and compute the true nearest key for each by exhaustive search.
3. Train a small regression network that maps a query vector to its best key, and a router that scores clusters.
4. Compare recall against latency, counting the network's own cost, as Apple did.

Metric to report: recall at 95% and 99%, latency per query including the network,
fraction of the database scanned, and recall on a held-out query set that differs from training.

The last line matters most. A router that wins on the training distribution and loses on new questions is not an improvement.

Limits to keep in mind

• Vendor-reported results. The numbers are from Apple's paper and blog, on Natural Questions with Apple's setup.
• Fixed database. The networks are trained for a fixed set of keys. Changing the corpus means retraining.
• Known query distribution. The method is built for stable query patterns. Apple says it tolerates mild shift, and gives no figures for large shift.
• Model size. The L models are 20% of the database size, which is a real memory cost for large key sets.
• Retrieval is not answer quality. Faster, more accurate retrieval helps, but the post does not measure end-to-end RAG answers.

Related reading

• Embeddings and vector search: the complete guide
• Semantic vs vector vs hybrid search
• RAG context injection pipeline design
• RAG vs agentic RAG
• Google TurboVec vector search in Rust

Details are accurate as of November 5, 2026. Results are as reported by Apple and may change in later versions of the paper.