Google DeepMind has added Tool Retrieval to the Gemini API, in preview. An agent can now keep a large library of tools on standby and load a tool's parameter schema only when a task needs it. Google's own announcement says the feature is "designed to reduce input tokens and keep tool selection accurate as your agent scales." It works in the Interactions API with Gemini 3.6 and later, including all Gemini 4 models. The announcement is by product manager Lucia Loher and research scientist Nora Kassner.
This post explains how defer_loading and tool_search work, what the pricing means, how server-side and client-side retrieval differ, and which numbers Google has not given.
TL;DR: Gemini API tool retrieval at a glance
| Question | Answer |
|---|---|
| What is it? | A way to hide tool schemas until the model asks for them |
| Where does it work? | Gemini Interactions API, preview |
| Which models? | Gemini 3.6 and later, including all Gemini 4 models |
| How do I turn it on? | Add a tool_search tool and set defer_loading: true on held-back tools |
| What does the model see? | Each deferred tool's name and description, not its parameter schema |
| Do I build an index? | Not for server-side search; Google runs the search |
| Is there an extra fee? | No; standard model token rates apply |
| Does prefix caching still work? | Yes, per Google, because deferred schemas sit outside the start of the prompt |
| Is there a benchmark? | No percentage or accuracy number in the announcement |
Why tool retrieval exists: the schema tax
Every function you give a model comes with a schema: the name, a description, and a JSON definition of each parameter. Send 5 tools and the cost is small. Send 80 and every request carries all 80 schemas, whether the task needs one of them or none.
Google names three problems with that. Input token counts rise. Latency rises. And tool selection gets less accurate as the list grows, because the model has more near-duplicate options to pick between. Developers who connect agents to several MCP servers feel this first, since a single server can expose dozens of tools.
Tool retrieval attacks the first two directly and aims at the third. The model starts with a short menu. It pulls the full definition of a tool only when it has decided it needs it.
How defer_loading and tool_search work
The server-side flow, as Google describes it, has four parts:
- You add a
tool_searchtool to the request. - You set
defer_loading: trueon every tool you want to hold back. - The model sees each deferred tool's name and description, but not its parameter schema.
- When the model needs a tool, it asks for it by name. The API loads the schema, and the model emits the function call. All of this happens within one request.
You can watch it happen. interaction.steps shows three entries: tool_search_call, tool_search_result, and function_call. That trace is the main debugging aid: if the model never loads the tool you expected, you can see whether it searched and what came back.
Gemini runs the search on the server, so there is no index to build or host. Tools you use constantly can stay undeferred, which keeps them one step away.
Deferring a whole MCP server
For remote MCP servers the shortcut is one flag. Setting defer_loading: true on an mcp_server tool defers every tool that server exposes. That matters because an MCP server's tool list is often not under your control. You connect, and you inherit whatever the server publishes. We covered the risks of that in our look at MCP tool provider misbinding, and a deferral flag does not change the trust question: the model still sees the names and descriptions of deferred tools, so those strings are still prompt input from the server.
Server-side versus client-side retrieval
Google offers two modes. The table summarizes the trade-off.
| Mode | Who runs the search | Index needed | Extra round trips | Best for |
|---|---|---|---|---|
| Server-side (default) | Gemini | No | None, one request | Large static tool libraries |
Client-side ("execution": "client") | Your app | Yours | One per search | Own registry, embedding index, per-user permissions |
In client mode you declare tool_search with "execution": "client". Your application performs the lookup and returns matching schemas in a function_result. The cost is one round trip per search. The gain is control: tools do not need to be declared upfront, so you can expose new ones mid-conversation, and you can filter by what a given user is allowed to call.
That second point is the practical reason to use client mode. If your agent serves many users with different permissions, a server-side search over one big catalog cannot know who is asking. Your own lookup can. The same concern, not letting the model choose the identity or the scope, appears in our piece on Bedrock Knowledge Base ACLs on AgentCore Gateway.
What does it cost?
Google states there is no separate charge. You pay the standard input and output rates for the model. Three details matter for your bill:
- Enabling
tool_searchadds a small, fixed block of input tokens to every request. - Each schema the model loads is billed as input for the rest of the conversation.
- Savings depend on whether the schemas you leave unloaded outweigh the ones that get loaded.
The announcement says that is typical for large libraries, and recommends measuring against your own traffic. That is honest advice and also a gap: there is no published figure for tokens saved, latency change, or accuracy gain. The only "50+" in the post is the number of deferred functions in a code example, not a result.
A rough way to think about it: if a task needs 3 of 60 tools, you pay for the fixed tool_search block plus 3 schemas instead of 60. If a task touches 40 of 60 tools, deferral saves little and adds search steps. Agents that call a few tools per task out of a long catalog win. Agents that use most of their tools every turn do not.
For pricing context on the models this runs on, see our coverage of the Gemini 4 Neon launch, price and benchmarks.
Does prefix caching still work?
Google says yes. Deferred schemas sit outside the start of the prompt, so the cached prefix is not disturbed when a schema is loaded mid-conversation. This is the detail that makes deferral compatible with prompt caching. A naive approach, which edits the tool list every turn, changes the prefix and throws the cache away.
The loaded schema is still billed as input for the rest of the conversation, as noted above. So caching keeps your stable prefix cheap; it does not make a loaded schema free.
How this fits Google's agent push
Tool retrieval lands in the Interactions API, which Google calls its primary interface for Gemini models and agents. That API already mixes built-in tools such as Google Search and code execution with your own functions in one request. Deferral is the next logical piece: once one request can hold many kinds of tools, you need a way to stop paying for the ones you are not using.
It also sits next to other agent plumbing Google has shipped lately. See our notes on Gemini managed agents with self-hosted sandboxes, and on how Gemini 4 Neon demos use tools in practice.
What is still unproven
Be careful with three claims.
Accuracy. Google says the feature helps "keep tool selection accurate as your agent scales." That is a design goal in the announcement, not a measured result. There is no benchmark. A model that must decide which tool to load, from a name and one-line description, can fail at that step. If two deferred tools have vague, similar descriptions, the model may load the wrong one or none at all.
Latency. Fewer input tokens should help. But each search adds a tool_search_call and tool_search_result step. In client mode it adds a full round trip. Net latency depends on your catalog size and how many searches a task needs.
Preview status. This is a preview. Behavior, limits and pricing can change. Do not move a production agent onto it without a fallback.
How to try it: a practical checklist
- Pick a model: Gemini 3.6 or later, including Gemini 4.
- Take your current tool list and split it. Keep the 3 to 5 tools used on most turns undeferred.
- Mark the long tail
defer_loading: true, and addtool_searchto the request. - Write sharp tool descriptions. The model chooses from name and description alone, so "Search customer invoices by date range" beats "Invoice helper."
- Log
interaction.steps. Count how often the model searches, which tools it loads, and whether it ever fails to find one. - Compare input tokens and task success on the same set of test tasks with and without deferral. That is the measurement Google tells you to run.
- If you need per-user permissions or tools that appear mid-conversation, test client-side search.
If you build agents for a living or a team, our guide to Claude Code token efficiency and prompt caching covers the same cost thinking from the Anthropic side.
Common questions searchers ask
Is this the same as lazy loading? The idea is similar: pay for a thing only when you use it. Here the "thing" is a tool schema inside the model's context, not a script in a web page.
Does it work with Gemini 2.5 or older? Google lists Gemini 3.6 and later. It does not list older models.
Can I use it outside the Interactions API? The announcement describes it only for the Interactions API.
Is it safe? Deferral reduces tokens. It does not add a security boundary. The model still sees names and descriptions of every deferred tool, and a loaded tool still runs with whatever permissions you gave it. Treat tool descriptions from third-party servers as untrusted input.
The bottom line
Tool retrieval is a small, practical feature with a clear use: agents with long tool catalogs. It costs nothing extra beyond tokens, keeps prefix caching intact, and offers a server mode that needs no infrastructure and a client mode that gives you control. What it does not offer yet is a number. Until Google or independent testers publish token, latency and accuracy results, treat the savings as a hypothesis to test on your own traffic.
Related reading
- Gemini 4 Neon launch, price and benchmarks
- Gemini 4 Neon use cases and demos
- Gemini managed agents with self-hosted sandboxes
- What is MCP? The complete guide
- MCP tool provider misbinding and agent security
- Bedrock Knowledge Base ACLs on AgentCore Gateway
- Claude Code token efficiency and prompt cache guide
Details reflect Google's announcement of November 19, 2026. The feature is in preview and may change.
