OpenAI asks whether frontier language models can do medicinal chemistry reasoning, not just caption molecules. On November 20, 2026 it announced MedChemBench — 99 graded problems built from published programs where improving one drug property often worsens another.
The release sits beside adversarial distillation safeguards and internal research acceleration metrics as OpenAI tries to show where automation helps science and where it still hallucinates structures.
TL;DR: scores OpenAI reports at xhigh reasoning
| Model (xhigh) | Overall credit (99 problems) | OpenAI-highlighted strength | Weak slice |
|---|---|---|---|
| GPT-6.2 Astra | 84.5% | Structure plus evidence explanation | Selection and numeric prediction |
| GPT-6 Astra | 83.3% | 95.4% structure component credit | About half credit on selection and prediction |
| GPT-6.1 Sol | 82.0% | Competitive overall | Illustrated ring failures in legacy tiers |
| GPT-5.6 Sol | 58.2% | Lagging frontier by OpenAI table | Failed 18-atom ring reconstruction in demo |
Higher reasoning settings generally raise scores in OpenAI''s plots, but large gaps between models remain.
What MedChemBench measures
Molecule nodes with one highlighted atom, suggesting structure-first grading
OpenAI groups each problem into four scored components:
- Molecular structures — correct atom connectivity and stereochemistry from drawings and specified groups.
- Scientific reasoning — explanations tied to supplied experimental evidence.
- Ranking and selection — ordering or choosing compounds that meet stated goals.
- Property predictions — numeric and categorical agreement with withheld or quoted measurements.
Problems snapshot a point in a medicinal chemistry campaign: target biology, program goals, compounds tested so far, and selected experimental readouts. That mimics how medchem teams decide what to synthesize next, not how Twitter demos caption a SMILES string.
The suite draws on 117 sources and 761 structures, so overlap with training corpora is plausible. OpenAI explicitly flags prior exposure as unruled-out.
Why drug properties trade off
Medicinal chemists routinely fight multi-objective tension. A potent kinase hit may absorb poorly or clear too fast in liver microsomes. Models that optimize one headline metric without the others mislead executive readouts.
OpenAI''s PRMT5 inhibitor vignette shows compound 23 blocking PRMT5 at lower cellular concentration while 24 shows better permeability and slower hepatic breakdown in withheld assays. Models must connect structure edits to competing readouts, the same jagged pattern OpenAI documents for coding agents.
Structure reconstruction is the gate
Folded protein ribbon hinting at spatial chemistry reasoning
Before property talk, the model must assemble the right molecule. One task preserves an 18-atom ring while attaching two groups. OpenAI reports GPT-6 Astra reconstructed it correctly in 10 of 10 xhigh attempts, while GPT-5.6 Sol shrank the ring to 15 atoms and GPT-5.6 Luna split disconnected fragments.
Those errors are not cosmetic. Wrong graphs send every downstream potency prediction off-cliff. Teams evaluating vision-language chemistry tools should treat graph validity as a hard gate, similar to unit tests before merge.
Component breakdown and token scaling
OpenAI plots overall credit versus output tokens across 51 model-and-setting combinations from GPT-5.6 onward, with 95% uncertainty intervals. A companion chart separates component curves, showing structure and explanation rising faster than selection and measurement prediction.
That shape suggests RL-friendly verifiable subtasks (did you draw the graph?) may saturate before judgment-heavy portfolio picks. It rhymes with external METR judgment-task gaps reported the same day, though MedChemBench tasks remain algorithmically scored.
What MedChemBench is not
OpenAI''s limitations section is explicit:
- No claim that models invent approved drugs
- No multi-turn lab agent with instruments
- No guarantee that high benchmark credit changes portfolio decisions
Compare to DOE Genesis open science models or Anthropic''s Genesis Mission credits: government programs fund access, while MedChemBench funds measurement.
How builders can use the benchmark
Pharma ML teams can treat MedChemBench as a regression suite when swapping tokenizers or vision encoders. Failures on ring size or stereochemistry should block deployment even if aggregate accuracy looks flat.
Model hosts should publish per-component dashboards, not one leaderboard number. OpenAI''s component split shows selection remains the business-risk slice.
Educators can use PRMT5-style tradeoff prompts to teach why AI-for-science hype needs assay context.
Links to OpenAI''s wider November science push
The same week OpenAI posted chip design with AI models and continues Rosalind trusted-access tiers for high-risk biology. MedChemBench is the public, published-data lane — lower governance friction, but also lower proof of wet-lab impact.
Evaluation hygiene checklist
| Step | Why it matters |
|---|---|
| Freeze reasoning effort settings | OpenAI compares models at shared xhigh labels |
| Log SMILES and graph edits | Detect silent structure drift between turns |
| Hold out entire paper series | MedChemBench problems cluster by publication |
| Run human medchem review on failures | Models may sound plausible while wrong on units |
| Do not conflate credit with ROI | Partial credit masks dangerous tail errors |
National-policy and safety context
Drug-discovery assistance sits in the same policy perimeter as cyber and bio distillation fights. OpenAI''s November distillation post notes cyber capabilities migrating to students. Chemistry reasoning could follow similar paths if traces leak from allowed programs.
Labs publishing open weights should pair capability posts with usage monitors, echoing OpenAI frontier risk training controls.
Walkthrough: how a single problem is framed
OpenAI''s post walks through compound 17, where models assemble a large ring system from a central scaffold plus substituents. The grading function is unforgiving: a wrong ring size changes inchi keys and every downstream QSAR-style guess.
That design mirrors how medchem teams actually work in discovery:
- Medicinal chemists propose analogs from SAR tables in papers.
- Computational chemists sanity-check graphs before docking or MD.
- Program leaders ask for explicit tradeoff language, not a single IC50 boast.
MedChemBench forces models to do all three rhetoric layers in text, even though it does not run robots or mass spec.
Token spend versus accuracy
OpenAI''s Figure 2 plots overall credit against tokens, including reasoning tokens. Scores rise with compute, but marginal gains shrink on selection tasks. Teams budgeting API spend for science assistants should expect nonlinear bills: the last 5 percentage points may cost more tokens than the first twenty.
Compare with OpenAI''s $2,400 median daily agent spend figure for coding researchers — chemistry benches may show similar heavy-tail spend if users crank reasoning effort to max on every compound panel.
Comparison to adjacent public benchmarks
MedChemBench is narrower than general science Q&A sets but deeper per compound than many SMILES-only leaderboards. It complements:
- Physics and math formal benchmarks OpenAI discussed in Yang-Mills Lean work
- Chip design tasks in OpenAI''s November 18 silicon post
- Enterprise chemistry copilots vendors sell without publishing task definitions
Without public task JSON, external labs cannot reproduce scores exactly. OpenAI''s paper link is the reproducibility anchor.
Responsible communications for startups
If your pitch deck says MedChemBench proves AI drug discovery, you are ahead of OpenAI''s own disclaimer. Safer claims:
- Models parse structures from literature-style prompts at high rates on this suite
- Models still miss portfolio judgments that humans make with incomplete assays
- Benchmark credit is partial; tail failures can be toxicology-relevant
Regulators and partners will ask for wet-lab confirmatory data, not LLM percentages.
Classroom and workshop uses
Instructors can assign students to:
- Reconstruct compound 17 by hand from the same prompt text
- Compare human SAR narrative to model narrative
- Identify where partial credit would hide a fatal graph error
That pedagogy aligns with explainx.ai workshop style: verify graphs before trusting prose.
Future updates OpenAI teases
The November post closes by saying MedChemBench should support continued progress as models advance. Watch for:
- New reasoning tiers on GPT-6.x lines
- Additional component weighting if selection improves
- Possible tool-using extensions (not in v1 scoring)
We will update this page if OpenAI ships a revision with new task counts or public leaderboards.
Procurement questions for vendor bake-offs
Enterprise buyers evaluating chemistry copilots should ask vendors for MedChemBench component splits, not a marketing screenshot. Request:
- Raw JSONL outputs on a holdout subset they did not train on
- Error taxonomy on graph failures versus unit failures
- Whether vision inputs match OpenAI''s drawing format or proprietary sketches
If a vendor refuses, treat headline accuracy as unaudited.
Safety and export-control awareness
Medicinal chemistry assistance can touch controlled precursor knowledge depending on jurisdiction. OpenAI places this release in the research transparency bucket, not a weapons-design claim. Still, infosec teams should log which employees run long chemistry chains on frontier APIs, similar to cyber dual-use policies in OpenAI distillation safeguards.
Paper and reproducibility path
OpenAI''s announcement links Read the paper from the same page. Teams reproducing scores should download that PDF for exact rubric weights, partial-credit formulas, and any holdout splits. Without the paper hash pinned in your MLOps repo, you cannot defend a vendor bake-off outcome six months later.
Document:
- Model ID and reasoning-effort knob per attempt
- Temperature and seed if exposed
- Prompt hash for each of the 99 tasks once JSON releases
Integration with cheminformatics stacks
Most builders will not call ChatGPT for every analog. They will pipe RDKit, OpenBabel, or enterprise ELN exports into an API. MedChemBench instead tests literature reasoning, closer to how a medicinal chemist reads a patent table than how a docking pipeline scores poses.
That gap means high MedChemBench credit does not imply high docking RMSD or MD stability. Use the benchmark where it measures narrative SAR quality, not physics engines.
Educator and workshop notes
Workshop facilitators can run a blind review: students grade model paragraphs before seeing model IDs. Disagreement between human medchem TAs and model scores illustrates judgment-heavy evaluation — the same theme METR highlighted on AI R&D tasks.
