Terminal-Bench-Science: The Benchmark That Deflates Coding-Agent Hype

Every frontier AI model just got dramatically worse at its job — or, more precisely, the industry finally built a benchmark that measures the job it was never actually good at. Terminal-Bench-Science 0.1, launched by Stanford University and the Laude Institute on August 27-28, 2026, tests AI agents on 70 tasks pulled directly from working scientists' own research pipelines. The result: Claude Opus 5, the best model on the leaderboard, resolves just 30.0% of tasks. Every single model evaluated drops by more than 10 percentage points compared to its score on Terminal-Bench, the general coding-agent benchmark from the same team.

That gap is the headline. If you have been reading Terminal-Bench 2.1 or 3.0 scores as a proxy for "how good is this agent at R&D, data science, or technical research," Terminal-Bench-Science says: stop. A model that clears 80%+ on general terminal tasks can still fail 70-97% of real scientific workflows. This post covers the leaderboard, why the gap exists, what it costs to run, and what it means for anyone evaluating agents for science or technical domains — a companion piece to our guide on how to read AI benchmarks without getting fooled.

TL;DR

The 70-task leaderboard

Every model ran three independent trials per task across all 70 tasks. Resolution rate is the share of tasks an agent actually completed correctly.

Two things jump out. First, no model clears a third of the tasks — even the leader, Claude Opus 5, fails 70% of real research workflows. Second, Anthropic and OpenAI models occupy every top slot except one — Grok 4.6 ties GPT-5.6 Sol for second place specifically in engineering sciences (see below), and GLM 5.3 is the only open-weight model to crack double digits anywhere on the leaderboard.

Why the score collapses: Terminal-Bench vs. Terminal-Bench-Science

This is the part that matters most for builders. The same models were already scored on Terminal-Bench 2.1 and 3.0 — the general-purpose coding-agent benchmark from the same Laude Institute team — and the numbers are not close:

Every model evaluated on both benchmarks drops by more than 10 percentage points moving to Terminal-Bench-Science — and for most models in this table, the real gap is 60+ points. The Laude Institute team describes this as deliberate: Terminal-Bench-Science tasks were calibrated during review specifically to challenge the newest frontier models, not to be solvable out of the box.

The practical reason the gap is this wide: real research tasks demand domain expertise (interpreting an astronomy dataset correctly, choosing the right statistical test, knowing when a simulation has converged), tolerance for genuine ambiguity (scientists don't write unit-tested specs the way software tickets do), and correctness that a human domain expert has to actually validate — not just "the tests pass." A coding agent that's excellent at satisfying a well-specified terminal task can still be lost the moment the task is "reproduce this ecology model's inference and tell me if the result is statistically sound."

This is the exact failure mode our how-to-read-AI-benchmarks guide warns about: a benchmark score is the output of a model, a dataset, a harness, and a task distribution — and a score from one distribution does not automatically transfer to another. Terminal-Bench-Science is a clean, quantified proof of that warning, aimed squarely at anyone who has been extrapolating from general coding benchmarks to scientific or R&D use cases. It pairs well with our audit of inflated and gamed benchmark claims and the receipts behind Goodhart's law in AI evaluation — Terminal-Bench-Science isn't a contamination story, but it's the same lesson: read the fine print on what a benchmark actually measures before you generalize from it.

Cost and token efficiency: the Pareto frontier

Resolution rate alone doesn't tell you what an agent costs to run at scale. The benchmark also tracked total evaluation cost and token usage across all 70 tasks (three trials each):

• GPT-5.6 Sol matches Claude Fable 5's resolution rate at less than a third of the cost — $4.2k total eval cost versus Fable 5's $14.2k.
• Claude Opus 5 reaches the highest resolution rate of any model at $7.0k total cost — cheaper than Fable 5 despite scoring higher.
• GPT-5.6 Luna, Kimi K3, and GPT-5.6 Terra sit at the low-cost end of the Pareto frontier — cheap to run, but with correspondingly low resolution rates (3.3%, 7.1%, and 8.6% respectively).
• On tokens, Claude Fable 5 matches GPT-5.6 Sol's performance using about a quarter fewer tokens — 6.4B versus 8.4B tokens across the eval.
• Only Kimi K3 and Claude Opus 5 appear on both the cost-Pareto and token-Pareto frontiers — the two models that are efficient by both measures simultaneously.

The takeaway for model selection: raw resolution rate and cost-efficiency rank differently. If you're picking a model for scientific-agent workloads at volume, GPT-5.6 Sol's cost-to-accuracy ratio is the standout data point on this leaderboard, even though Claude Opus 5 wins on raw capability. This is the same tradeoff we cover in our broader Fable 5 vs. Grok 4.6 vs. GPT-5.6 Sol comparison — accuracy leaderboards and cost leaderboards are genuinely different rankings, and conflating them is a common evaluation mistake.

Domain breakdown: where each model actually leads

Terminal-Bench-Science splits its 70 tasks across five science domains:

Task types span scientific data analysis, statistical inference, simulation, optimization, theorem proving, image reconstruction, signal processing, inverse problems, sensor calibration, model fitting, classification, and scientific machine learning — a far wider net than the shell-scripting and repo-navigation tasks that dominate general coding benchmarks.

Claude Opus 5 leads in most domains, including Earth sciences at 45.8% — the single highest domain-model score on the entire leaderboard — plus the top spot in Life sciences and Physical sciences. The one exception is Mathematical sciences, where Claude Fable 5 (33.3%) and GPT-5.6 Sol (31.4%) take the top two spots, ahead of Opus 5. In Engineering sciences specifically, Grok 4.6 ties GPT-5.6 Sol for second place at 14.8%, doing so at lower cost and fewer tokens than its Anthropic and OpenAI competitors in that domain — the only domain where a non-Anthropic, non-OpenAI model matches the pace.

How selective is Terminal-Bench-Science, really?

The 70 tasks that shipped in v0.1 are the survivors of a genuinely brutal review funnel:

Those proposals came from 376 contributors across 22 countries — not model developers or data vendors, but the scientists who actually run these workflows. Each surviving task passed a multi-stage gauntlet: propose, domain-reviewer discussion and approval, implementation as a PR, a scientific-validity and realism check from domain reviewers, a separate technical review of construction and verification, and a final "bar raiser" quality check before merge. That funnel is the strongest signal that these are not toy tasks dressed up as "scientific" — the 7.6% survival rate is closer to a competitive journal's acceptance rate than a typical benchmark's inclusion bar.

The benchmark was built by the Terminal-Bench/Harbor team in collaboration with domain experts across life, physical, Earth, mathematical, and engineering sciences from research institutions worldwide, and is hosted by Stanford University and the Laude Institute in partnership with Stanford AI Lab (SAIL), Stanford HAI, Stanford AI Measurement Science (AIMS), the NSF AI Institute for Foundations of Machine Learning (IFML), the Allen Institute, and Ai2. Project lead advisors are Ludwig Schmidt and Sanmi Koyejo; the announcement was written by Steven Dillmann. Support and compute credits came from the Laude Institute's Slingshots program, Snorkel AI's Open Benchmarks Grants, the 2077AI Open Source Foundation, UniPat AI, Modal, and API credits from Bespoke Labs, Anthropic, Google, Moonshot AI, SpaceXAI, and Z.ai.

Terminal-Bench-Science is explicitly designed as a continuous, evolving benchmark, not a one-off release — future versions will add tasks, retire ones that become saturated or underspecified, and stay calibrated to whatever the current frontier can do. Version 0.2 is already in motion, with a pull-request deadline of October 5, 2026.

What this means for evaluating your own agents

If you're building or buying agents for R&D, data science, or any technical/scientific domain, three things follow directly from this launch:

1. Don't extrapolate coding-agent scores to scientific-reasoning tasks. A model that tops Terminal-Bench or SWE-bench is not automatically strong at your domain — the 10+ point gap here is a floor, not a ceiling, and most models in this dataset lost 60+ points.
2. Weight cost against capability separately. GPT-5.6 Sol matching Fable 5's accuracy at a third of the cost is the kind of number that should change a procurement decision, not just a leaderboard screenshot. Run your own cost-per-resolved-task math the way our price-per-token coverage argues, rather than trusting sticker price alone.
3. Build or find a domain-specific eval before you ship. If Terminal-Bench-Science's own survival funnel (7.6% of proposals became tasks) is any signal, generic benchmarks massively under-specify what "hard" looks like in a real scientific workflow — the same lesson our agent evaluation with real repos piece makes for software specifically.

Related reading

• PRAXIST Beta: Sapient Intelligence cumulative research agents (Aug 28, 2026)
• Terminal-Bench 2.0: The AI Agent Benchmark That Actually Matters
• How to Read an AI Benchmark and Not Get Fooled
• AI Benchmarks: The Complete Guide
• The AI Benchmark Numbers That Need Fact-Checking
• Goodhart's Law Comes for Every Benchmark You Trust
• Google DeepMind's first double-blind AI evaluation: a structural fix for benchmark contamination
• Fable 5 vs. Grok 4.6 vs. GPT-5.6 Sol vs. Qwen3-8-Max Comparison
• Anthropic VirBench: Why Biological Agents Need Deterministic Tools
• Google AI Scientist at ICML 2026: ScientistOne and Chain-of-Evidence

Model names, resolution rates, and cost figures reflect Terminal-Bench-Science v0.1 as announced August 27-28, 2026. Scores will shift as the benchmark evolves toward v0.2 (PR deadline October 5, 2026) — check the official benchmark site for the current leaderboard before citing these numbers in your own evaluation.