Specification gaming and Goodhart's law explained why any metric used as a target eventually stops measuring well. That was the theory. On July 28, 2026, ACM's BLOG@CACM published "Goodhart's Law Comes for Every Benchmark You Trust" by Alex Williams — and it is not theory anymore. It is receipts: contamination rates, error-audit percentages, and a controlled experiment showing exactly how much a leaderboard score can be manufactured.
This post walks through that evidence. If you have ever picked a model off a leaderboard, or cited a benchmark score in a procurement deck, the numbers below apply to you directly.
TL;DR — the questions people actually ask
| Question | Answer |
|---|---|
| Do public benchmarks leak into training data? | Yes, measurably — GSM1k found up to 13% accuracy drop on fresh, unseen problems |
| Are MMLU's answer keys reliable? | No — 6.49% overall error rate across 5,700 re-annotated questions; 57% flawed in virology |
| Can LMArena be gamed without a fixed test set? | Yes — best-of-N private submission; Meta tested 27 Llama-4 variants before publishing one |
| Is 17 points of Arena score just noise? | Yes — two identical checkpoints under different names diverged by 17 points |
| Did LMArena dispute this? | Partly, and fairly — some points land, but the core asymmetry finding stands |
| What should I do instead of trusting a leaderboard? | Build a 20-50 question private eval from your own data |
How a benchmark's own defense mechanism failed
BIG-bench, the collaborative benchmark suite, embeds a unique "canary" string specifically so researchers can filter it out of training data or audit whether a model was trained on it. When OpenAI prepared its GPT-4 technical report, contamination checks found BIG-bench had been swallowed into GPT-4's training data anyway — the model could reproduce the canary string on request, and the results had to be excluded.
The mechanism designed to catch cheating did not survive contact with web-scale crawling. That is the opening image of the CACM piece, and it sets up the rest: if a benchmark built with contamination detection as a first-class feature still gets contaminated, what happens to the benchmarks that never planned for it?
Goodhart's law, formalized
Charles Goodhart's 1975 observation about British monetary policy — "when a measure becomes a target, it ceases to be a good measure" — has since been formalized by Manheim and Garrabrant into four distinct variants of "Goodharting." Two dominate AI evaluation in 2026:
- Regressional Goodharting — the test quietly becomes part of the training data.
- Adversarial Goodharting — the evaluation process itself gets gamed.
The specification-gaming post covers the mechanism and product-team implications in depth. What follows is the empirical evidence for both variants, published in 2026.
Evidence 1: GSM1k proves regressional Goodharting with numbers
Researchers at Scale AI (Zhang et al., 2024) commissioned 1,205 fresh, human-written grade-school math problems matched in difficulty to GSM8K — the standard elementary math reasoning benchmark — specifically designed so no model could have seen them during training.
| Metric | Finding |
|---|---|
| Accuracy drop, initial evaluation | Up to 13% |
| Accuracy drop, final released set | Up to 8% |
| Model families showing systematic overfitting | Phi and Mistral, across almost all sizes |
| Correlation: verbatim reproduction ↔ accuracy gap | Spearman's r² = 0.36 |
That last row is the important one. The more likely a model was to reproduce GSM8K problems verbatim, the larger its accuracy gap between GSM8K and GSM1k. That is not a coincidence — it is a direct statistical signature of memorization standing in for mathematical reasoning.
Evidence 2: the answer keys themselves are wrong
Even a perfectly uncontaminated benchmark is only as good as its answer key. Gema et al. re-annotated 5,700 MMLU questions across all 57 subjects and estimated a 6.49% overall error rate — wrong answer keys, ambiguous phrasing, unanswerable questions. In the virology subset specifically, 57% of analyzed questions were flawed.
Correcting those errors changed model rankings. That means leaderboard position was, in part, a function of which models happened to "correctly" reproduce a wrong answer key rather than which models actually knew the material.
A separate, broader audit — "Fantastic Bugs" (Truong et al.) — applied measurement-theoretic statistical analysis across nine widely-used benchmarks. Expert review confirmed up to 84% of the top flagged questions had substantive flaws. GSM8K itself, the benchmark GSM1k was built to check against, carries an estimated ~5% error rate of its own.
As the CACM article puts it: "We have been ranking billion-dollar systems, to a decimal place, on answer keys we never proofread."
For the fuller catalog of what's currently saturated, contaminated, or still discriminating — MMLU, GPQA, SWE-bench, Terminal-Bench, and the rest — see the AI benchmarks complete guide, which this post extends with the concrete contamination and error-rate numbers behind the saturation story it already documents.
Evidence 3: LMArena and the Leaderboard Illusion
LMArena (Chatbot Arena) was built specifically to resist contamination — live human A/B votes, no fixed test set, a Bradley-Terry ranking model. "The Leaderboard Illusion" (Singh et al., 2025 — Cohere Labs, Stanford, Princeton, MIT, Ai2, and others) analyzed roughly 2 million Arena battles across 243 models and 42 providers and found it can still be gamed structurally.
| Finding | Detail |
|---|---|
| Private pre-launch testing | Meta tested 27 different Llama-4 variants on Arena before publishing only the best-performing one |
| Noise floor, controlled experiment | Two identical model checkpoints submitted under different names diverged by 17 points |
| Data asymmetry, top providers | Top 2 proprietary providers each received an estimated 19-20% of all battle data |
| Data asymmetry, open models | 83 open-weight models combined received under 30% of battle data |
| Arena-distribution fine-tuning | Up to 112% relative gain on ArenaHard, while actual MMLU performance slightly declined |
The best-of-N pattern breaks the statistical assumption the Bradley-Terry model depends on: that each submitted model is one honest, independent sample. Submit 27 variants and publish the best one, and you are not sampling — you are selecting. The 17-point identical-checkpoint experiment quantifies exactly how much pure ranking noise that selection can exploit.
Separately, Meta was reported to have submitted a Llama 4 variant specifically optimized for "conversationality" that outranked the model it actually shipped to the public — a gap between what wins the leaderboard and what ships to users.
LMArena's rebuttal — presented fairly
LMArena disputed parts of the framing, and some of it lands:
- The identical-checkpoint score divergence falls within overlapping statistical confidence intervals.
- The private-testing policy has been publicly disclosed since March 2024 and was open to any provider with sufficient capacity — not secret.
- The 112% relative-gain figure comes specifically from ArenaHard, not the live Arena itself.
None of that disputes the core finding: well-resourced labs systematically used best-of-N submission while smaller and open labs submitted once, and the ranking model's statistical assumptions do not account for that asymmetry. No deliberate villain is required for Goodhart's law to apply here — it is a structural incentive problem, not a conspiracy.
For a companion case where the flaw was in the tasks rather than the ranking mechanism, see the SWE-bench Pro audit, where OpenAI found roughly 30% of tasks broken — overly strict tests, underspecified prompts — and retracted its own recommendation to adopt the benchmark. Different failure mode, same underlying lesson: a headline pass rate is not evidence until someone has audited what's underneath it.
Why this keeps happening
The incentive chain is short and nobody in it is directly rewarded for interrogating what the number means:
Leaderboard position → press coverage → fundraising decks → enterprise procurement decisions.
The term "benchmarketing" has emerged for this pattern. Benchmark saturation compounds it — MMLU is now passed at roughly 90% by frontier models, and HumanEval and HellaSwag are effectively ceilinged, as detailed in the complete benchmarks guide. When every model aces one benchmark, the field moves to a new one, which then gets contaminated or saturated in turn, on an accelerating cycle. The Terminal-Bench 2.0 methodology is one attempt to keep that cycle honest with sandboxed, task-based scoring rather than static multiple choice — worth reading alongside this piece for how a newer benchmark tries to design contamination resistance in from the start.
What the evidence says to actually do
Private, refreshed test sets — the only mechanism-level fix
Questions that were never on the public web cannot leak into training data, and rotating them means memorizing this year's set does not help next year. GSM1k is the template; ImageNetV2 and CIFAR-10.1 served the same purpose for vision research a decade earlier, revealing that classifiers had overfit the originals. The tradeoff: expensive to build, and protection is temporary — private sets "rot," just more slowly than public ones.
Contamination detection is table stakes, not sufficient
Sun et al. tested 20 proposed mitigation strategies across 10 models and 5 benchmarks and found none significantly improved contamination resistance over doing nothing, and none stayed faithful to the original benchmark's intent while resisting contamination. N-gram overlap checks and perplexity probes should run by default — just don't treat a clean scan as proof.
For deployment decisions: evaluate on your own data
The article's most practical, least-glamorous recommendation: a 20-50 question mini-benchmark built from a team's actual tickets, contracts, or queries reportedly beats any public leaderboard for predicting real-world performance, because no model has been trained on that specific workload. If pairing this with an LLM-as-judge to scale grading, the judge's own known biases — toward position, verbosity, and its own outputs — must be actively controlled, or a team ends up Goodharting its own private eval too.
For shared, public leaderboards
The Leaderboard Illusion authors propose four structural fixes: prohibit score retraction, require disclosure of every variant tested, cap private submissions per provider, and equalize sampling across providers. The CACM article is skeptical whether any leaderboard can hold that line while its biggest users — frontier labs — are also its most influential participants.
The line worth remembering
"Any number that decides funding, rankings, or procurement will be optimized, by gradient descent or by a marketing department, and usually by both... Treat every public score as a marketing claim that happens to carry decimal places. Hold the private questions back. And when a model aces your test, assume first that it has seen the answers."
Summary
The theory in specification gaming and Goodhart's law now has hard numbers behind it: GSM1k's up-to-13% accuracy drop on fresh problems, MMLU's 6.49% error rate, and the Leaderboard Illusion's 17-point noise floor and 27-variant best-of-N submission pattern. None of this requires a villain — it is Goodhart's law playing out at benchmark scale, structurally, across labs. The actionable response is unglamorous but real: build a small private eval on your own data, treat contamination checks as necessary but not sufficient, and read every public leaderboard score as a marketing claim with decimal places attached.
Related on explainx.ai
- Specification gaming, Goodhart's law, and the metrics that lie about AI
- AI Benchmarks in 2026: The Complete Guide to MMLU, GPQA, SWE-bench, and Beyond
- OpenAI Audits SWE-Bench Pro: ~30% of Tasks Broken
- Terminal-Bench 2.0: The AI Agent Benchmark That Actually Matters
- DeepSWE — prior SWE-Bench Pro scrutiny
- How to build your own enterprise AI benchmark
Sources: ACM BLOG@CACM — Goodhart's Law Comes for Every Benchmark You Trust, July 28, 2026 · GSM1k / Zhang et al. 2024 · The Leaderboard Illusion / Singh et al. 2025
Statistics and findings reflect the sources as published through July 28, 2026; benchmark curators and leaderboard operators may revise methodology after this date — verify current practices before citing scores in procurement decisions.