explainx.ai350K+ learn from us
TrendingAI News TodayExplainx for Business
Pricing
explainx.ai

Upskill in AI — 16 free pathways, live workshops & bootcamps, and 50+ courses from practitioners. Plus the skills, tools, and MCP servers to practice on.

follow us

follow on google

Add explainx.ai as a preferred source

corporate training

support@explainx.ai

get started

Find your pathTake Free Evaluation

community

Join the community

learn

mind: share how you thinkpathways — start freeworkshopsbootcampscoursescompare Explainxcertificationsmock testsexplainx universitycorporate traininglearn skills & mcp

discover

skillsmcp serversexplainx mcpestacktoolsmdx readeragentsllmsdesignsdictionarypeopleagi trackerfelony benchranks

company

aboutvisionmissionteaminstructorsteach on explainxpartnershipscommunityhackathonscareers

content

daily AI newsstate of AI — live resultsblogreleasespromptsgeneratorsresource libraryfor LLMsexplainx.ai kids

solutions

all solutionsdeveloper upskillingmarketing upskillingproduct manager upskillingleadership upskilling

newsletter · weekly

Get AI news, tools, and insights in your inbox.

supportcontactprivacytermsdata rightshow we create contentsubmission guidelines

© 2026 AISOLO Technologies Pvt Ltd

explainx.ai

On this page

  • TL;DR: Gemini API tool retrieval at a glance
  • Why tool retrieval exists: the schema tax
  • How defer_loading and tool_search work
  • Server-side versus client-side retrieval
  • What does it cost?
  • Does prefix caching still work?
  • How this fits Google's agent push
  • What is still unproven
  • How to try it: a practical checklist
  • Common questions searchers ask
  • The bottom line
  • Related reading
← Back to blog

explainx / blog

Gemini API Tool Retrieval: Defer Loading and tool_search Explained

Gemini, Google DeepMind, AI Agents, MCP, Developer Tools

Part of Google Gemini and DeepMind

Google added tool retrieval to the Gemini Interactions API: set defer_loading, add tool_search, and load tool schemas only when needed. How it works.

Nov 19, 2026·9 min read·Yash Thakker
add explainx.ai
go deep
Gemini API Tool Retrieval: Defer Loading and tool_search Explained

Google DeepMind has added Tool Retrieval to the Gemini API, in preview. An agent can now keep a large library of tools on standby and load a tool's parameter schema only when a task needs it. Google's own announcement says the feature is "designed to reduce input tokens and keep tool selection accurate as your agent scales." It works in the Interactions API with Gemini 3.6 and later, including all Gemini 4 models. The announcement is by product manager Lucia Loher and research scientist Nora Kassner.

This post explains how defer_loading and tool_search work, what the pricing means, how server-side and client-side retrieval differ, and which numbers Google has not given.

TL;DR: Gemini API tool retrieval at a glance

table · 2 cols
QuestionAnswer
What is it?A way to hide tool schemas until the model asks for them
Where does it work?Gemini Interactions API, preview
Which models?Gemini 3.6 and later, including all Gemini 4 models
How do I turn it on?Add a tool_search tool and set defer_loading: true on held-back tools
What does the model see?Each deferred tool's name and description, not its parameter schema
Do I build an index?Not for server-side search; Google runs the search
Is there an extra fee?No; standard model token rates apply
Does prefix caching still work?Yes, per Google, because deferred schemas sit outside the start of the prompt
Is there a benchmark?No percentage or accuracy number in the announcement
Daily briefFrom Yash · 350K+ students taught

Catch up on AI

One AI story explained in plain English, plus one thing to try. Every day, free. Unsubscribe anytime.

Why tool retrieval exists: the schema tax

Every function you give a model comes with a schema: the name, a description, and a JSON definition of each parameter. Send 5 tools and the cost is small. Send 80 and every request carries all 80 schemas, whether the task needs one of them or none.

Google names three problems with that. Input token counts rise. Latency rises. And tool selection gets less accurate as the list grows, because the model has more near-duplicate options to pick between. Developers who connect agents to several MCP servers feel this first, since a single server can expose dozens of tools.

Tool retrieval attacks the first two directly and aims at the third. The model starts with a short menu. It pulls the full definition of a tool only when it has decided it needs it.

How defer_loading and tool_search work

The server-side flow, as Google describes it, has four parts:

  1. You add a tool_search tool to the request.
  2. You set defer_loading: true on every tool you want to hold back.
  3. The model sees each deferred tool's name and description, but not its parameter schema.
  4. When the model needs a tool, it asks for it by name. The API loads the schema, and the model emits the function call. All of this happens within one request.

You can watch it happen. interaction.steps shows three entries: tool_search_call, tool_search_result, and function_call. That trace is the main debugging aid: if the model never loads the tool you expected, you can see whether it searched and what came back.

Gemini runs the search on the server, so there is no index to build or host. Tools you use constantly can stay undeferred, which keeps them one step away.

Deferring a whole MCP server

For remote MCP servers the shortcut is one flag. Setting defer_loading: true on an mcp_server tool defers every tool that server exposes. That matters because an MCP server's tool list is often not under your control. You connect, and you inherit whatever the server publishes. We covered the risks of that in our look at MCP tool provider misbinding, and a deferral flag does not change the trust question: the model still sees the names and descriptions of deferred tools, so those strings are still prompt input from the server.

Server-side versus client-side retrieval

Google offers two modes. The table summarizes the trade-off.

table · 5 cols
ModeWho runs the searchIndex neededExtra round tripsBest for
Server-side (default)GeminiNoNone, one requestLarge static tool libraries
Client-side ("execution": "client")Your appYoursOne per searchOwn registry, embedding index, per-user permissions

In client mode you declare tool_search with "execution": "client". Your application performs the lookup and returns matching schemas in a function_result. The cost is one round trip per search. The gain is control: tools do not need to be declared upfront, so you can expose new ones mid-conversation, and you can filter by what a given user is allowed to call.

That second point is the practical reason to use client mode. If your agent serves many users with different permissions, a server-side search over one big catalog cannot know who is asking. Your own lookup can. The same concern, not letting the model choose the identity or the scope, appears in our piece on Bedrock Knowledge Base ACLs on AgentCore Gateway.

What does it cost?

Google states there is no separate charge. You pay the standard input and output rates for the model. Three details matter for your bill:

  • Enabling tool_search adds a small, fixed block of input tokens to every request.
  • Each schema the model loads is billed as input for the rest of the conversation.
  • Savings depend on whether the schemas you leave unloaded outweigh the ones that get loaded.

The announcement says that is typical for large libraries, and recommends measuring against your own traffic. That is honest advice and also a gap: there is no published figure for tokens saved, latency change, or accuracy gain. The only "50+" in the post is the number of deferred functions in a code example, not a result.

A rough way to think about it: if a task needs 3 of 60 tools, you pay for the fixed tool_search block plus 3 schemas instead of 60. If a task touches 40 of 60 tools, deferral saves little and adds search steps. Agents that call a few tools per task out of a long catalog win. Agents that use most of their tools every turn do not.

For pricing context on the models this runs on, see our coverage of the Gemini 4 Neon launch, price and benchmarks.

Does prefix caching still work?

Google says yes. Deferred schemas sit outside the start of the prompt, so the cached prefix is not disturbed when a schema is loaded mid-conversation. This is the detail that makes deferral compatible with prompt caching. A naive approach, which edits the tool list every turn, changes the prefix and throws the cache away.

The loaded schema is still billed as input for the rest of the conversation, as noted above. So caching keeps your stable prefix cheap; it does not make a loaded schema free.

How this fits Google's agent push

Tool retrieval lands in the Interactions API, which Google calls its primary interface for Gemini models and agents. That API already mixes built-in tools such as Google Search and code execution with your own functions in one request. Deferral is the next logical piece: once one request can hold many kinds of tools, you need a way to stop paying for the ones you are not using.

It also sits next to other agent plumbing Google has shipped lately. See our notes on Gemini managed agents with self-hosted sandboxes, and on how Gemini 4 Neon demos use tools in practice.

What is still unproven

Be careful with three claims.

Accuracy. Google says the feature helps "keep tool selection accurate as your agent scales." That is a design goal in the announcement, not a measured result. There is no benchmark. A model that must decide which tool to load, from a name and one-line description, can fail at that step. If two deferred tools have vague, similar descriptions, the model may load the wrong one or none at all.

Latency. Fewer input tokens should help. But each search adds a tool_search_call and tool_search_result step. In client mode it adds a full round trip. Net latency depends on your catalog size and how many searches a task needs.

Preview status. This is a preview. Behavior, limits and pricing can change. Do not move a production agent onto it without a fallback.

How to try it: a practical checklist

  1. Pick a model: Gemini 3.6 or later, including Gemini 4.
  2. Take your current tool list and split it. Keep the 3 to 5 tools used on most turns undeferred.
  3. Mark the long tail defer_loading: true, and add tool_search to the request.
  4. Write sharp tool descriptions. The model chooses from name and description alone, so "Search customer invoices by date range" beats "Invoice helper."
  5. Log interaction.steps. Count how often the model searches, which tools it loads, and whether it ever fails to find one.
  6. Compare input tokens and task success on the same set of test tasks with and without deferral. That is the measurement Google tells you to run.
  7. If you need per-user permissions or tools that appear mid-conversation, test client-side search.

If you build agents for a living or a team, our guide to Claude Code token efficiency and prompt caching covers the same cost thinking from the Anthropic side.

Common questions searchers ask

Is this the same as lazy loading? The idea is similar: pay for a thing only when you use it. Here the "thing" is a tool schema inside the model's context, not a script in a web page.

Does it work with Gemini 2.5 or older? Google lists Gemini 3.6 and later. It does not list older models.

Can I use it outside the Interactions API? The announcement describes it only for the Interactions API.

Is it safe? Deferral reduces tokens. It does not add a security boundary. The model still sees names and descriptions of every deferred tool, and a loaded tool still runs with whatever permissions you gave it. Treat tool descriptions from third-party servers as untrusted input.

The bottom line

Tool retrieval is a small, practical feature with a clear use: agents with long tool catalogs. It costs nothing extra beyond tokens, keeps prefix caching intact, and offers a server mode that needs no infrastructure and a client mode that gives you control. What it does not offer yet is a number. Until Google or independent testers publish token, latency and accuracy results, treat the savings as a hypothesis to test on your own traffic.

Related reading

  • Gemini 4 Neon launch, price and benchmarks
  • Gemini 4 Neon use cases and demos
  • Gemini managed agents with self-hosted sandboxes
  • What is MCP? The complete guide
  • MCP tool provider misbinding and agent security
  • Bedrock Knowledge Base ACLs on AgentCore Gateway
  • Claude Code token efficiency and prompt cache guide

Details reflect Google's announcement of November 19, 2026. The feature is in preview and may change.

Spotted something out of date? Let us know.
Daily briefFrom Yash · 350K+ students taught

Catch up on AI

One AI story explained in plain English, plus one thing to try. Every day, free. Unsubscribe anytime.

  • ✓ Free
  • ✓ One email a day, no spam
  • ✓ Unsubscribe anytime
Yash Thakker

Written by

Yash Thakker

Yash is an AI expert with over 300K learners. Join his workshops →

View Yash Thakker in People in AI →

Related posts

Nov 5, 2026

Web Answer on Amazon Bedrock AgentCore: Cited Answers in One Tool Call

Amazon Bedrock AgentCore now offers Web Answer, a managed gateway connector that plans searches, reads pages and returns a synthesized, source-cited answer in a single tool call. Here is how it works, the three setup steps, the limits, and when to pick it over Web Search.

Nov 5, 2026

Executor 2 Beta: One Open-Source MCP Gateway for Claude Code, Cursor and Codex (MIT, 4.3k Stars)

Executor, the open-source integration layer from Rhys Sullivan and UsefulSoftwareCo, entered its 2.0 beta this week, with new builds on November 3, 4 and 5, 2026. It puts MCP servers, OpenAPI specs and GraphQL APIs behind one catalog with allow, approve or block policies, so Claude Code, Cursor and Codex share the same tools. Here is what it does, how to install the beta, and what to check before you trust it with your accounts.

Oct 28, 2026

Google Connected Apps Platform: One Build for Search and Gemini via MCP

Google has put a developer front door on its connected apps: one cloud MCP server or Android AppFunctions integration, one console, and a choice of Search, Gemini or both. Here is what is confirmed, what is not, and what builders should do.