Google Cloud published a piece by Balaji Subramaniam, a DevRel Engineer on the team, on September 2, 2026, working through context caching for coding-agent harnesses on Google ADK 2.0 and the Gemini Enterprise Agent Platform. It crossed 38,400 views in a day, which is a lot for a post about prompt serialization.
It deserves the attention, because it does something most caching content does not: it publishes measured token counts across three distinct multi-agent topologies, run in Python 3.11 sandboxes on Cloud Run, rather than asserting that caching is good and moving on.
It also reports its headline as transmitted token reduction, which is not the same thing as your bill. Both numbers are worth having, so below is the mechanism, the four operating rules, Google's measurements, and the cost arithmetic the post leaves as an exercise.
TL;DR
| Question | Answer |
|---|---|
| What's the problem? | Models are stateless. The harness re-sends the whole static prefix every turn. |
| How bad? | 37,500-token prefix + 300-token traceback × 5 turns = ~190,000 prompt tokens. Ten turns ≈ 400,000. |
| What breaks caching? | Any mutable value before or inside the static prefix. The match is byte-for-byte from token zero. |
| Measured saving | 74.6%–79.5% fewer transmitted tokens across three topologies. |
| Actual bill saving | Closer to 56%–59% — cached reads still cost 0.25x and creation costs 1.0x. |
| Minimum prefix | 32,768 tokens. Below that, don't bother. |
| Minimum turns | Three. Two turns can't amortize the 1.0x creation charge. |
| The one design rule | Dynamic variables go at the end of the prompt. Never at the beginning. |
The arithmetic that makes this necessary
Large language models are stateless. They take an input sequence and return generated text; they remember nothing. The agent harness — the thing managing runtime state, sandbox execution, and prompt preparation — holds the conversational memory, and it does that by re-serializing the full context window on every model invocation.
Follow one typical coding loop. Static reference code is 37,500 tokens. Each turn appends a 300-token error traceback.
| Turn | Transmitted | Cumulative billed |
|---|---|---|
| 1 | 37,800 | 37,800 |
| 2 | 37,800 | 75,600 |
| 3 | 37,800 | 113,400 |
| 4 | 37,800 | 151,200 |
| 5 | 37,800 | 189,000 |
Five turns, 189,000 prompt tokens, of which roughly 187,500 are the same 37,500 tokens uploaded five times. A ten-turn refactor against a large repository approaches 400,000 prompt tokens. The network re-uploads identical files; the server re-tokenizes them.
This is the same accumulation curve we worked through from the Anthropic side in what actually costs you tokens in a Claude Code session and in context window pricing, decoded. Different vendor, identical physics.
The prefix-breaking trap
Here is the part that turns a nice optimization into a silent failure mode, and the reason prefix invariance is a design constraint rather than a tuning tip.
Context caching requires an exact, byte-for-byte token match starting from token zero of the prompt. Not "mostly the same." Not "the same file contents." The same tokens, from the first one.
So if the harness places any mutable runtime metadata before or inside the static text — a timestamp, a session UUID, a turn counter, a "current time is..." line in the system preamble — the prompt hash changes. The server cannot match the request to the precomputed cache. It falls through to a cache miss, and the full prompt is charged at standard rates.
The post's rule is one sentence and it is the single most important line in it:
Dynamic variables belong at the end of the prompt, never at the beginning.
The failure is silent. Nothing errors. Your latency is a bit worse and your bill is unchanged, and unless you are watching cached-token counts specifically, you will conclude caching "didn't help much" when in fact it never engaged. If you are building your own harness rather than using someone else's — the territory covered in our minimal agent harness walkthrough and the context, prompt, loop and harness engineering stack — this is the first thing to get right and the easiest to get wrong.
The correct prompt order
1. Tool definitions ← static, cache this
2. System prompt ← static, cache this
3. Repository / spec context ← static, cache this (the 37k tokens)
──────────────── cache boundary ────────────────
4. Dynamic suffix ← traceback, compiler error, this turn's ask
Google's reference implementation enforces this with two components: a CachePayloadBuilder that structurally isolates immutable prompt headers from dynamic execution suffixes so prefix corruption is impossible by construction, and a ContextCacheManager that owns the lifecycle — computing content hashes to reuse existing active caches, extending TTL when needed, and dispatching cached inference requests.
The naming matters less than the shape: one object that can only build a compliant payload, one object that owns cache lifetime. If your harness lets any code path concatenate a string into the prefix, you will eventually break it.
The measured results
Google ran three multi-agent topologies in Python 3.11 sandboxes on Cloud Run. All three used a static prefix in the 37k–39k token range.
| Scenario | Turns | Uncached input | Cached transmitted | Transmitted reduction |
|---|---|---|---|---|
| Multi-target batch modernization — five Python 2.7 modules against a 37,659-token monorepo prefix | 5 | 190,139 | 39,503 | 79.46% (150,636 eliminated) |
| Adversarial SQLi red/blue debate — exploiter vs fixer agents against a 38,367-token OWASP spec | 4 | 154,008 | 38,907 | 74.69% (115,101 eliminated) |
| Multi-file dependency graph refactor — four interdependent microservice layers against a 38,441-token ORM SDK | 4 | 154,479 | 39,156 | 74.56% (115,323 eliminated) |
Two things stand out.
The topology barely matters. A batch of independent tasks, an adversarial two-agent debate, and a cascading dependency refactor all land in the same 75–79% band. What determines the saving is the ratio of static prefix to dynamic suffix and the turn count — not how the agents are wired to each other.
The dynamic suffixes are tiny. In scenario 1, five turns of real agent work produced 1,844 tokens of new content against a 37,659-token prefix. In scenario 2 it was about 540 tokens. That ratio — roughly 1:20 to 1:70 — is the actual shape of iterative coding work, and it is why caching is not a marginal optimization here.
The number Google didn't publish: what it costs
The headline is transmitted reduction — bytes over the wire and server-side re-tokenization avoided. Real, but not your invoice.
Cached content reads are billed at a 75% discount, i.e. 0.25x the standard prompt rate, and cache creation is charged at 1.0x (the post says as much when it explains why sub-three-turn runs cannot amortize it). So the billed volume is:
creation (prefix × 1.0)
+ reads (prefix × 0.25 × remaining turns)
+ dynamic suffix (× 1.0)
Running that on Google's own figures:
| Scenario | Uncached | Effective billed (cached) | Real cost reduction |
|---|---|---|---|
| Batch modernization (5 turns) | 190,139 | ~77,200 | ~59% |
| Red/blue debate (4 turns) | 154,008 | ~67,700 | ~56% |
| Dependency refactor (4 turns) | 154,479 | ~68,000 | ~56% |
A 56–59% cut in input cost is an excellent result. It is just not 79%, and the gap is worth internalizing before you present a savings estimate to anyone who will hold you to it. The transmitted figure is the right metric for latency and bandwidth; this one is the right metric for the finance conversation — the same distinction we drew in token budget planning and execution.
Note also that the saving improves with turn count — the 1.0x creation charge is fixed while the 0.25x reads keep accruing against a 1.0x counterfactual. At 10 turns the same scenario 1 prefix lands closer to 70% off. Long loops are where this pays.
The four rules, as a decision tree
Google's operational guidance is refreshingly short. Applied in order:
| Condition | Decision | Why |
|---|---|---|
| Static context < 32,768 tokens | Don't cache | Transmission cost is minor; cache management overhead isn't justified |
| Turn count < 3 | Don't cache | One- and two-turn calls can't amortize the 1.0x creation cost |
| Prompt headers change every cycle | Don't cache — refactor first | You'll never get a hit. Isolate the static files, then revisit |
| Iterative test-and-repair against a large repo | Always cache | This is the case the mechanism exists for |
The third row is the one teams skip. "Refactor prompt construction to isolate static files before attempting to cache" is not a suggestion you can defer — an un-isolated prefix means caching is disabled in practice while appearing enabled in configuration.
Porting this to other providers
The mechanism is universal; the constants are not.
| Google Cloud (as described) | Anthropic | |
|---|---|---|
| Cached read | 0.25x standard prompt rate | ~0.1x |
| Cache write | 1.0x | up to 2x |
| Minimum useful prefix | 32,768 tokens | Much lower, model-dependent |
| Cache lifetime | Managed TTL, extendable | 5 min on API key, 1 hour on subscription |
| What breaks it | Any change before/inside the prefix | Same, plus model and effort switches |
Two consequences of that table. First, Anthropic's cheaper reads and pricier writes push the break-even toward more turns with a smaller prefix, while Google's cheaper writes and pricier reads favor fewer turns with a bigger prefix. Second, the engineering discipline is identical either way: isolate the static prefix, append dynamics at the tail, own cache lifetime in one place. Build the harness that way and you can retarget providers without redoing the work. Our prompt caching cost-optimization guide covers the cross-provider version in more depth.
Honest limitations
- These are vendor benchmarks. Google measured Google's platform on synthetic workloads it designed. The methodology is disclosed and reproducible, which is more than most, but it is not third-party.
- Synthetic workloads are unusually favorable. Real coding sessions include file reads mid-loop, tool results of unpredictable size, and users changing their minds — all of which enlarge the dynamic suffix relative to these 540–1,844 token tails.
- No dollars, no latency figures. Ratios and token counts only. Wall-clock improvement from skipping re-tokenization is not reported.
- 32,768 is a rule of thumb, not a hard threshold, and it will move as pricing does.
- Cache management is real operational surface. TTL extension, hash computation, eviction, and cache-miss observability are code you now own and have to monitor. Below the threshold, that overhead is the whole reason not to bother.
- ADK 2.0 specifics may not survive contact with your stack.
CachePayloadBuilderandContextCacheManagerare that implementation's names; the design pattern ports, the API does not.
Related on explainx.ai
- What actually costs you tokens in a Claude Code session
- Prompt caching and LLM cost optimization
- Context window pricing, decoded
- Context, prompt, loop — the harness engineering stack
- Building a minimal agent harness
- Token budget planning and execution
- Context engineering vs prompt engineering
- Loop engineering for coding agents
Token counts, discount rates, and threshold guidance reflect Google Cloud's post as published September 2, 2026. The cost-reduction figures in the "what it costs" section are our own arithmetic on Google's published token counts using the stated 0.25x read and 1.0x creation rates — they are estimates, not vendor figures. Pricing and thresholds change; verify against current Gemini Enterprise Agent Platform documentation before budgeting against them.