Google AI Scientist at ICML 2026: ScientistOne, Chain-of-Evidence, and the End-to-End Research Pipeline

On July 7, 2026, Google Research posted on X inviting ICML attendees to "Experience AI Scientist" — a multi-agent system meant to automate the end-to-end scientific research pipeline. The booth slot was 6:00pm at #B206, with Jinsung Yoon and Rui Meng listed as contacts, tagged #ICML2026.

That marketing name — "AI Scientist" — lines up with an ICML Expo Talk Panel from the day before: The End-to-End AI Scientist: Automating Discovery and the Research Pipeline (July 6, 2026, Hall D2), same authors. The published research system with the matching three-stage architecture is ScientistOne — Towards Human-Level Autonomous Research via Chain-of-Evidence (Google Cloud AI Research, arXiv 2605.26340).

This post maps the booth narrative to verified paper claims only. No invented benchmark numbers, no conflating separate Google research products.

TL;DR — what people are asking

Three names, one booth — keep them straight

Google's autonomous-research story is not one monolith:

The ICML panel abstract also mentions ScholarPeer (automated peer review with a "historian" literature approach for novelty) and PaperBanana (methodology figure generation). Those appear in the talk abstract; ScientistOne's paper evaluates manuscript quality with ScholarPeer but centers verification on Claim Verifier + CoE Audit inside the pipeline.

The pipeline — Problem Investigator → Discovery → Paper + Claim Verifier

The architecture Google showed at ICML matches ScientistOne Figure 1 and the booth diagram below:

Stage 1: Problem Investigator

Inputs: task definition + seed papers.

Process (from arXiv §4.1 and Appendix B):

1. Build a citation graph via scholarly database queries.
2. Filter to an elite pool (~500 papers) with tier scores for methodology relevance and problem alignment.
3. Run multi-round investigation — Librarian, Researcher, and SubdomainWriter agents across ~3 rounds targeting ~100 structured paper notes and 5–15 research directions.
4. Emit an Experiment Brief (research landscape, concrete experiment plan, 25–40 references traceable to read PDFs).

Design intent: citations must come from retrieved full text, not model memory. In CoE Audit, systems without retrieval grounding hit up to 21% hallucinated references; ScientistOne reports 0/337.

This is the opposite of "prompt and pray" — closer to Mollick-style specs: define the task, define evidence, define how you will test claims.

Stage 2: Discovery Engine + Parallel Explore-Exploit

Ideator: reads the PI brief, generates candidate approaches, scores novelty and feasibility, expands top ideas into proposals.

The booth diagram labels conservative vs unconventional generation tracks before proposals fan out. The arXiv text describes novelty/feasibility scoring and distribution across branches — treat C/U as visual shorthand for diverse ideation, not a separate published benchmark metric.

Parallel Explore-Exploit (PEE): proposals scatter into parallel branches (B1…B4 in the diagram; paper uses configurable B branches). Each branch runs:

• Solve — Solver agent (sandboxed code execution, experimental log).
• Evaluate — task-specific evaluator scores each submission.
• Audit — specification-violation checks filter metric-gaming or invalid submissions (CoE I2).

After each iteration, top-K branches survive; remaining slots refill from fresh ideation on winners. Loop for I iterations × B branches. A best-run selector picks the highest-scoring clean solution, then runs ablations.

What happens to failed experiments? They are expected and discarded — pruned branches, spec-violation drops, and low scores never reach Stage 3. Feedback (distillation, metrics) can inform the next ideation round. This is loop engineering at research scale: explore widely, exploit winners, stop on violations.

Stage 3: Paper Writer + Claim Verifier

Paper Writer — five stages (arXiv §4.3, Appendix B.4):

Claim Verifier: extract claims → verify against declared evidence (numeric tolerance on logs, bibliography + abstract entailment for citations, log overlap for methods) → refinement pass rewrites or removes failures → final_paper.pdf only if no blocking violations remain.

That directly targets the "slop" failure mode: pretty PDFs where numbers, citations, and code disagree.

Chain-of-Evidence (CoE) and the Integrity Audit

ScientistOne's thesis: verifiability is architectural, not a post-hoc spell-check.

CoE requires every claim to trace to code, data, or literature through the full run.

CoE Integrity Audit (post-hoc, cross-system) runs four checks on deliverables (paper.tex, solution code, references.bib):

Headline results (75 papers, five systems, five ADRS tasks — project site):

† Sakana cross-system I2/I4 comparisons are confounded by scaffolding in submitted code — see paper.

Solver performance still matches or beats human expert baselines on ADRS tasks; generalization tests include MLE-Bench (e.g., Gold on 3D Object Detection where DeepScientist scored 0.0000) and Parameter Golf SOTA as of the April 27, 2026 cutoff.

What the Claim Verifier does — and does not — catch

X threads asked whether Claim Verifier flags accidental rediscovery — e.g., reinventing a 1987 result.

What it does verify (per paper):

• Citation keys resolve to real publications retrieved into the bibliography.
• Abstract entailment for specific citation assertions.
• Numerical claims match evaluator logs within tolerance.
• Method descriptions align with experimental logs and (via CoE I4) solution code.

What it explicitly does not guarantee:

ScholarPeer serves as a scalable proxy for review quality but does not replace human expert evaluation … CoE Integrity Audit … is itself limited to structural integrity, not scientific novelty or significance.

So: if the Problem Investigator never retrieved the 1987 paper, the pipeline will not magically flag "you rediscovered Smith et al. 1987." If that paper is in the PI brief and the draft claims novelty anyway, Critic and ScholarPeer-style review are the layers meant to push back — still imperfect.

For the agent vs agency debate — whether tools like this are "co-scientists" or autonomous agents — see Eric Xing's Critique of Agent Model: marketing labels outrun what the harness actually controls.

"Auto research race" — who else is in the pool?

July 2026 social chatter compared Google and Anthropic. Map that to what shipped publicly:

The race is real on harness + verification, not on replacing peer review overnight. Agent harness engineering already showed scaffolding beats model swaps on Terminal-Bench; ScientistOne argues the same for research integrity.

ICML panel extras — ScholarPeer and PaperBanana

The ICML panel abstract adds components beyond the core ScientistOne diagram:

• ScholarPeer — automated peer review; panel mentions a "historian" approach to check novelty against historical literature (ScholarPeer arXiv:2601.22638, cited in ScientistOne for review scores).
• PaperBanana — automated methodology figure synthesis (multi-modal challenge).

ScientistOne's published evaluation uses ScholarPeer ratings (40% accept rate vs 13% best baseline) but treats it as a proxy, not ground truth.

Honest limitations — read before the hype

From the ScientistOne paper's limitations section:

1. Domains — ADRS/MLE-Bench use deterministic evaluators. Wet lab, materials, pure theory need new verification logic.
2. Audit false negatives — flagged failures were human-verified; undetected integrity bugs certainly exist.
3. Seed variance — same task, different seeds → very different ScholarPeer outcomes (paper cites EPLB scores 1, 3, 8 across seeds).
4. Baseline depth — PI retrieves related work, but accepted papers still get dinged for shallow baseline comparisons vs human SOTA re-implementations.
5. Cost — full parallel search trees at B=20 width are compute-heavy; width helped TXN scores in scaling tables but Cloudcast/Prism saturated early.

What builders should steal

You do not need Google's token budget to apply the design pattern:

1. Provenance before prose — bind claims to log lines before LaTeX/Markdown export (agent markdown discipline).
2. Parallel explore, serial exploit — branch ideas, evaluate on golden harness, kill spec violations early.
3. Typed claim verification — numbers, citations, and methods need different checks, not one generic "review the PDF."
4. Cross-system audit — CoE Audit is a template for comparing your agent's papers to Sakana/DeepScientist-style outputs honestly.

Related on explainx.ai

• Jacobian conjecture and Fable 5 counterexample explained — a compact case where model-assisted discovery became independently checkable mathematics
• Agent harness engineering — Terminal-Bench & LangChain — why scaffolding beats raw model upgrades
• Ethan Mollick: specs, not prompting tricks — management framing for agentic research tasks
• What is loop engineering? — explore/exploit loops beyond one-shot prompts
• Adaption AutoScientist — automated model training — parallel "AutoScientist" name, different stack
• Eric Xing: agentic vs agentive — vocabulary for co-scientist marketing
• Anthropic J-space interpretability hub — Anthropic's July 2026 research lane (verification via mechanistic tools, not paper bots)
• Evans Nature study — AI flattens discovery — individual agent wins vs collective narrowing; incentives over architecture

Official sources

• ScientistOne project site
• ScientistOne arXiv HTML
• ICML 2026 — The End-to-End AI Scientist panel
• Google Research X post — July 7, 2026
• Co-Scientist — Google DeepMind blog
• MARS — Google Research publication

Architecture, audit tables, and benchmark numbers follow Google Cloud AI Research's ScientistOne paper and project site as of July 7, 2026. ICML booth branding ("AI Scientist") may differ from arXiv product naming; verify against primary sources before citing in academic or investor materials.