PaperOrchestra is Google Cloud AI Research's open-source multi-agent pipeline that turns experiment logs and bullet-point ideas into conference-formatted LaTeX — with automated literature review, section writing, refinement, and optional plotting. The google-research/paper-orchestra repo listed 1,034 GitHub stars when we fetched it on November 16, 2026, alongside an arXiv:2604.05018 citation in the README.
For ML engineers, it is a concrete reference architecture for decomposing "write the paper" into orchestrated agents — not a substitute for peer review, human authorship ethics, or replication culture.
TL;DR
| Component | Role |
|---|---|
| Outline agent | Structure from raw notes |
| Literature agent | Search, prioritize, integrate citations |
| Section writers | Draft body text per section |
| Refinement agent | Iterative polish |
| Plotting agent | Optional code-generated figures |
| Output | LaTeX under templates/ (e.g., CVPR 2025) |
Why Google open-sourced another agent framework
Research teams already use Claude for lattice work and GPT-6 Astra science blogging. The bottleneck shifted to manuscript assembly: related work, consistent notation, figure regeneration after ablations.
PaperOrchestra attacks that last mile with specialists instead of one monolithic prompt — the same pattern as Sakana multi-agent self-supervision but aimed at LaTeX delivery.
The README credits Yiwen Song, Yale Song, Tomas Pfister, and Jinsung Yoon at Google Cloud AI Research. The project site is yiwen-song.github.io/paper_orchestra/.
Pipeline architecture
A large cream ring routes three small green parcels outward — PaperOrchestra's specialized writer, literature, and refinement agents
PaperOrchestra advertises five coordinated stages:
- Outline generation from unconstrained pre-writing material.
- Literature review with search, prioritization, and integration into prose.
- Section writing aligned to the outline.
- Content refinement passes.
- Plotting (optional) from executable code.
A Streamlit demo lives under frontend/ for interactive runs — useful for lab demos, not production CI.
Compare to TradingAgents' multi-role finance simulators — both show role boundaries beat one chat thread, but PaperOrchestra targets static PDF artifacts.
Installation and environment
conda create -n paper_orchestra python=3.11
conda activate paper_orchestra
pip install -r requirements.txt
Create .env with keys the README lists:
SEMANTIC_SCHOLAR_API_KEYOPENAI_API_KEYand/orGEMINI_API_KEYor Vertex (VERTEX_AI_PROJECT,VERTEX_AI_LOCATION)- Optional SMTP fields if the demo emails finished PDFs to you
Dataset gap: README explicitly says evaluation data is not bundled — future release under datasets/. You supply --raw_materials_dir with your own logs.
CLI usage
Bash wrapper:
./paper_writing_cli.sh
Direct Python:
python paper_writing_cli.py \
--raw_materials_dir /path/to/raw/ \
--latex_template_dir /path/to/templates/cvpr2025/ \
--output_dir /path/to/output/ \
--use_plotting true
Key flags:
--writer_model_name— LLM for drafting and literature agents.--reflection_model_name— LLM for critique passes.--use_plotting— enable figure agent (default false).
Templates live under templates/; add a conference by creating a new subfolder with the .sty and skeleton TeX your venue requires.
Repository layout for contributors
| Path | Purpose |
|---|---|
methods/ | Agent logic and orchestration |
autoraters/ | Automated evaluation scripts |
utils/ | PDF helpers, API utilities |
templates/ | CVPR and other conference shells |
frontend/ | Streamlit UI |
autoraters/ signals Google intends measurable quality gates — align your internal adoption with Nvidia SkillEvaluator thinking for agent outputs.
Ethics, authorship, and review risk
Automated manuscripts raise venue policy and integrity questions:
- Disclose LLM assistance per journal and conference rules.
- Verify citations — retrieval agents can hallucinate papers or misquote abstracts.
- Do not submit unreviewed LaTeX — treat output as draft zero, not camera-ready.
Our AI-generated code license debate applies doubly when generated text ships in Apache-licensed repos with human names on the byline.
PaperOrchestra is not OpenAI's Orchestra Bio drug agents — naming collision only.
How it fits the multi-agent research stack
| Layer | Example in ecosystem |
|---|---|
| Experiment | Hy4, Qwen, internal baselines (frontier comparison) |
| Benchmark | VibePrevisBench gaming agents |
| Write-up | PaperOrchestra |
| Review | Human peers + autoraters in repo |
Teams building embodied AI criteria benchmarks may feed logs directly into --raw_materials_dir if they standardize markdown experiment diaries.
Streamlit demo workflow
The frontend/ README (linked from the main repo) describes a Streamlit interface for non-CLI users. Typical lab workflow:
- Drop markdown experiment notes into
raw_materials_dir. - Pick a LaTeX template matching your target venue folder.
- Watch agents stream section drafts (API costs accrue per reflection loop).
- Download compiled PDF and diff against human baseline.
Streamlit is ideal for PI review; CLI mode fits batch rewrites after ablation sweeps.
Model provider matrix
| Provider env var | Agent stages typically using it |
|---|---|
GEMINI_API_KEY / Vertex | Writer + literature (Google-native retrieval tools) |
OPENAI_API_KEY | Alternative writer or reflection |
SEMANTIC_SCHOLAR_API_KEY | Citation search rate limits |
You can assign cheaper models to reflection while reserving frontier models for related-work synthesis — but cheap reflection may miss factual errors in methods sections describing novel architectures like MMFlash speculative decoding.
How this differs from generic "write my paper" prompts
Single-shot ChatGPT paper requests collapse structure, citations, and figures into one context window. PaperOrchestra's separation means:
- Literature agents can restart retrieval without wiping methods drafts.
- Plotting agents can re-execute matplotlib (or similar) after hyperparameter changes.
- Refinement agents target section-level tone instead of whole-document rewrites.
That mirrors production agent design in envisioning agentic robotics surveys — specialized roles with orchestration, not one omnibus prompt.
Limits and what to test first
- Run without plotting on a toy template to validate TeX compile on your machine.
- Swap writer and reflection models — cost and quality trade off sharply on long related-work sections.
- Check Semantic Scholar rate limits with your API key tier.
- Do not assume CVPR template equals acceptance — formatting ≠ science.
Agent roles in more detail
Three grippers place different green shapes into one shallow tray — outline, literature, and reflection agents handing off LaTeX sections
Although the public README groups agents coarsely, the implementation under methods/ typically separates:
- Retrieval planning — what keywords and venues to query (Semantic Scholar API key in
.env). - Evidence ranking — which papers enter the related-work section versus appendices.
- Notation normalization — keeping symbols consistent across sections (failure mode in single-thread LLM writes).
- Reflection — a second model critiques drafts (
--reflection_model_name).
Reflection agents mirror Claude team code review patterns: a different model family can catch hedging or missing limitations paragraphs.
Templates and conference compliance
templates/cvpr2025 ships as an example; you can clone the folder for NeurIPS, ICML, or internal tech reports. PaperOrchestra does not verify you uploaded the latest official style files — maintainers must refresh .sty drops themselves.
Before submission:
- Compile with your local TeXLive.
- Run venue PDF tests (page limit, anonymization).
- Manually inspect bibliography keys — agents love duplicate cites.
Autoraters and evaluation gap
The autoraters/ directory hints at automatic quality scoring, but without the withheld datasets/ folder you cannot reproduce Google Cloud AI Research numbers locally. Treat autoraters as optional dev tooling, not peer review.
When comparing to VibePrevisBench or gaming-agent papers, decide whether automated writing helps camera-ready prose or only internal tech reports — many labs ban the former.
Relationship to Google's other agent releases
PaperOrchestra complements Google Stitch Skills (coding agents) and Gemini Robotics coverage elsewhere on explainx.ai — same company, different surface. None of these repos replace Vertex governance policies on customer data.
Responsible use checklist
- Human authorship — list who verified claims and ran experiments.
- Citation spot checks — open every DOI the literature agent inserted.
- Figure provenance — plotting agents execute code; archive notebooks.
- Dual submission policy — automated variants still count as prior work if posted online.
- Export controls — unrelated to LaTeX, but relevant if your logs include restricted benchmark numbers from defense-linked evals.
Future dataset release
When Google ships the promised datasets/ drop, rerun PaperOrchestra on their benchmark and compare to your in-house logs — until then, star the repo for architecture reference, not leaderboard scores.
Security and secrets handling
The .env file holds provider keys and optional SMTP credentials for demo email delivery. Treat generated output_dir folders like any research artifact:
- Scrub API keys from experiment logs before feeding
raw_materials_dir. - Redact patient identifiers or customer names from traces if your lab works on applied AI such as Apple health AI studies.
- Do not commit LaTeX outputs with embedded gen-AI watermarks to arXiv without venue policy review.
PaperOrchestra is not a confidential compute enclave — calls go to configured cloud APIs unless you wire local models yourself (not documented as first-class in the README).
Star velocity and why researchers noticed
The repo's +306 stars in a day spike tracks renewed interest in agentic science workflows after Edward Witten's Claude-assisted lattice note and OpenAI lattice science blogging. Teams want pipelines between experiment folder and arXiv source zip without hiring a dedicated technical writer for every ablation grid.
Expect forks that add internal citation styles (IEEE, company tech report templates) under templates/ — upstream encourages custom subfolders.
Integrating PaperOrchestra into a real lab calendar
A pragmatic rollout keeps humans in the loop at three gates: outline approval, related-work spot check, and figure recompilation after final ablations. Schedule the CLI overnight when API rate limits are looser, then let staff edit LaTeX manually the next morning.
Pair with internal git: store output_dir as an artifact, tag the commit hash of code used for results, and block auto-submission hooks from pushing to arXiv until a PI signs off — the same discipline you would apply to Sakana-style multi-agent training logs.
Repository structure and arXiv id match google-research/paper-orchestra README, November 16, 2026.
