Gemma 4 E2B at 108,000 Tokens per Second: What Google Claims Gemini 4 Argon Did

Google says its newest model, Gemini 4 Argon, has been used to make Google's own open Gemma models nearly three times faster. The claim sits in the Gemini 4 Argon launch post published October 13, 2026: a 2.85x end-to-end inference speedup on an Nvidia A100 through vLLM, taking Gemma 4 E2B from about 37,900 to over 108,000 tokens per second.

It is an interesting claim because it is not about Argon's own benchmark scores. It is about Argon being used as an engineer, writing the optimizations that make another model cheaper to serve. This post lays out what Google wrote, what is missing, and how to check it.

TL;DR: claimed vs verified

What did Google actually write?

The launch post lists several things Argon was used for. One of them, under the heading about making Gemma models nearly three times as fast, reads: "Using Argon, we achieved a 2.85x end-to-end inference speedup on an A100 via vLLM (accelerating throughput from ~37,900 to over 108,000 tokens per second on Gemma 4 E2B) with 100% token parity and zero quality drop in just a few days, a process that would usually take months." It adds that readers can check the upstream pull requests, "optimizing everything from attention tuning to speculative decoding," to see the exact changes.

The same post describes other uses: helping the Chrome team untangle roughly 6 million lines of legacy browser-feature code inside Chromium's 40-million-line codebase, and enzyme design where Argon helped identify five novel serine esterase designs that lab testing confirmed were active. Those sit alongside the speedup as examples of Argon doing multi-step engineering and science work. For the full launch story, prices and benchmarks, see our Gemini 4 Argon launch post, and for the community side, the top use cases.

Why a tokens-per-second number needs context

Throughput claims depend on the workload. A figure like 108,000 tokens per second on a single A100 is a very high number for a language model, but Gemma 4 E2B is a small model, and serving engines reach high aggregate throughput by batching many requests at once. Several things the post does not say would change how you read it:

• Batch size and concurrency. Aggregate throughput rises with more simultaneous requests. A single user does not see 108,000 tokens per second.
• Prompt and output lengths. Prefill-heavy and decode-heavy workloads behave very differently.
• Precision and settings. Quantization level, context length and sampling parameters change speed.
• Baseline. 37,900 tokens per second is the starting point; how well-tuned that baseline was determines how much of the 2.85x is new.
• Metric. "End-to-end" suggests the whole serving path, but the exact benchmark harness is not described in the post.

None of this makes the claim false. It means you should treat 2.85x as the relative improvement on Google's test and the absolute number as a ceiling for that setup.

What "100% token parity" does and doesn't tell you

Token parity means that, on the tested prompts, the optimized system emitted exactly the same tokens as the original. That is a stronger check than similar benchmark scores because it rules out silent quality loss from approximations. It also fits optimizations that preserve math, such as faster attention kernels or well-implemented speculative decoding, which accepts only tokens the main model would have produced. What it does not tell you is how many prompts were tested or whether parity held on edge cases like long contexts. Ask for the test set size if you plan to rely on it.

What "attention tuning" and "speculative decoding" actually are

Google's post names two categories of change, and it helps to know what each usually means in a serving engine like vLLM.

Attention tuning covers the work of making the attention step, the part of the model that looks back over the prompt and previous tokens, run faster on a specific GPU. That can mean better memory access patterns, fused operations that avoid writing intermediate results to memory, or kernels tuned to the A100's cache sizes. These changes keep the math the same, which is consistent with a claim of identical tokens.

Speculative decoding uses a small draft mechanism to guess several next tokens, then has the main model verify them in one pass. vLLM's speculative decoding docs describe it as theoretically lossless up to hardware numerics, with the caveat that floating-point precision and batch size can still cause small output variations. When guesses are right, you emit multiple tokens for the price of one verification step; when they are wrong, you discard them and fall back. Correctly implemented, the output is identical to normal decoding, because only tokens the main model would have chosen are accepted. That is why speculative decoding is compatible with a 100% token parity claim, and why it is a common route to large speedups on small models.

Seen this way, the headline is plausible as engineering: a stack of individually modest improvements to a small model's serving path that compound. What is novel, if the claim holds, is the speed at which the work was done and that the author was a model.

What people are asking

Does this make Gemma 4 E2B 2.85x faster on my laptop? No. The result is for an A100 data-center GPU under vLLM. Laptop and phone runtimes use different engines and hardware, and the same techniques may or may not have been ported.

Is 100,000-plus tokens per second realistic for a small model? For aggregate throughput on a data-center GPU with heavy batching, small models can reach very high numbers. The number says little about latency for one chat session.

Is Google grading its own homework? Partly. The model, the engine and the benchmark all come from, or were run by, people with an interest in the result. That is normal for launch posts, and it is why independent replication matters. The fact that the changes were submitted upstream to an open engine makes outside verification possible, and that is the best feature of the claim.

What does it mean for cost? If your serving bill is dominated by throughput on a small model, a 2.85x gain would cut cost per token by roughly that factor on the same hardware. That is a back-of-envelope reading, not a Google statement, and real savings depend on utilization.

Should I upgrade my vLLM install? Test first. Pin a version, rerun your own benchmark, and compare outputs. A throughput gain is only useful if your quality checks still pass on your data.

Why this matters even if you never use Gemma

Two reasons.

First, it is a data point on AI-assisted systems engineering. Inference optimization is a typical "months of expert work" task: profile, rewrite kernels, tune scheduling, validate. If a model can compress that to days, serving costs for open models could fall faster than hardware improves. That is a vendor claim on a single example, and the right reaction is interest plus a request for replication. Our earlier posts on inference speedups, such as the Magnitude engine claim of 2x faster than Ollama and speculative decoding with DFlash 2 drafters, show how often headline multipliers shrink under independent testing.

Second, it lands in an open-model serving stack that people actually run. Because the changes were sent upstream to vLLM, the benefit, if real, accrues to anyone serving Gemma through that engine rather than being locked in a Google product. For on-device and small-model use, see our coverage of Gemma 4 26B A4B on Mac with MLX.

How to test it yourself

1. Pin versions: the vLLM release that includes the merged pull requests and the matching Gemma 4 E2B checkpoint.
2. Use a fixed prompt set and record input and output lengths.
3. Measure aggregate throughput at several concurrency levels, and per-request latency at each.
4. Compare token-for-token outputs against the baseline version with the same seed and greedy decoding.
5. Report GPU model, driver, precision and context length alongside the numbers.

If your results land well below 2.85x on an A100, that is not evidence the claim is wrong; it is evidence your workload differs from Google's.

What would make this verified

A reproducible script from Google or the vLLM maintainers, the list and merge status of the pull requests, and an independent run on the same hardware. We will add an update if any of these appear. Until then, read the claim as a credible, specific, self-reported result: Google names the GPU, engine, model and parity condition, which makes it testable, but publishes no raw logs.

Related reading on explainx.ai:

• Gemini 4 Argon launch: benchmarks and pricing
• Top 10 Gemini 4 Argon use cases
• Gemini 4 Argon vs Opus 5.5 vs Grok 4.7 vs GPT-6 Astra
• Magnitude inference engine 2x claim checked
• DFlash 2 draft model speculative decoding in vLLM
• Gemma 4 26B A4B on Mac with MLX

Figures are from Google's October 13, 2026 Gemini 4 Argon launch post; they are self-reported and may be revised.