OpenAI introduced BioBasicsBench on November 6, 2026, a benchmark that asks AI agents to do something ordinary and hard: read the raw output of everyday wet-lab experiments. At the highest shared reasoning setting, GPT-6 Astra earned an average of 66.7% of the available credit, against 43.8% for GPT-5.6 Sol. The more useful part of the OpenAI post is the list of failures it publishes next to the wins.
The benchmark was built with Ligo, a biotech company focused on enzyme design. Its 100 tasks use gel images, colony plates, microscopy stacks, sequencing reads, plate-reader exports and videos of bench work. This is a different test from a question-and-answer quiz on biology facts.
TL;DR: what OpenAI announced
| Question | Answer |
|---|---|
| What is it? | A 100-task benchmark of routine wet-lab analysis, built by OpenAI with Ligo |
| What do agents get? | Raw data: gels, blots, plates, microscopy, FASTQ files, instrument exports, bench videos |
| Best score (xhigh effort) | GPT-6 Astra, 66.7% mean reward (CI 60.3 to 73.1%) |
| Previous generation | GPT-5.6 Sol, 43.8% (CI 37.6 to 50.0%) |
| Models tested | Seven OpenAI models, 37 settings, 18,500 results |
| Main caveat from OpenAI | Higher scores do not yet mean reliable judgment in the lab |
| Public code or data? | The post we read links none |
What does BioBasicsBench actually test?
The 100 tasks span five areas of routine work: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. Some tasks are simple, such as reading the molecular weight of a DNA sample on a gel. Others ask for a detailed assay analysis.
OpenAI states the motivation plainly. Everyday lab work "involves failed experiments, ambiguous images and conflicting observations." Benchmarks built from the cleaned-up charts that appear in papers can miss that mess. BioBasicsBench tries to keep it.
Each task checks several skills together: biological knowledge, interpreting images and other data, detecting errors and understanding the protocol. Agents return a structured answer with a conclusion, measurement or recommendation. A rubric then awards credit per criterion. That allows partial credit, so the benchmark can tell a fully correct analysis from one that reaches the right conclusion for weak reasons.
What kinds of data are in the tasks?
OpenAI lists the inputs by count. A task can include more than one kind, so the numbers overlap.
| Data type | Tasks | Examples |
|---|---|---|
| Gels and blots | 31 | Agarose DNA gels, SDS-PAGE, Western blots |
| Protocols, notes and specs | 27 | SOPs, lab notebooks, photographed handwritten notes |
| Tables and instrument exports | 20 | CSV and Excel files, raw flow cytometry (FCS) files |
| Bench, apparatus and reagent photos | 18 | Gel rigs, tube racks, pH strips, a freezer |
| Microscopy | 14 | Brightfield and fluorescence images, z-stacks, time-lapse |
| Instrument screens and plots | 13 | NanoDrop, plate reader and balance screens |
| Sequencing files | 12 | Long-read FASTQ, FASTA, GenBank maps |
| Protocol videos | 7 | Streaking, serial dilution, weighing |
| Agar plates and colonies | 7 | Transformation plates, blue/white screens |
| Reference sheets | 21 | Ladder charts, plasmid maps (supporting material) |
The spread matters. A model that only reads text cannot pass a task that needs a photo of a gel rig. This is why OpenAI ties the gains to "stronger native multimodal capabilities."
How did the models score?
A scoreboard with bars being redrawn, showing BioBasicsBench scores across seven OpenAI models
OpenAI evaluated seven of its own models. Rival lab models are not in the table. The figures below are from OpenAI's own results at the "xhigh" setting, the highest reasoning effort reported for all seven models.
| Model | Mean reward | 95% CI | Output tokens per task |
|---|---|---|---|
| GPT-6 Astra | 66.7% | 60.3 to 73.1% | 4,704 |
| GPT-6.1 Sol | 62.0% | 55.6 to 68.4% | 5,238 |
| GPT-5.6 Sol | 43.8% | 37.6 to 50.0% | 9,099 |
| GPT-6 Luna | 38.3% | 32.3 to 44.5% | 13,120 |
| GPT-5.5 | 35.4% | 29.7 to 41.0% | 11,135 |
| GPT-5.4 | 33.2% | 28.3 to 38.4% | 13,717 |
| GPT-5.2 | 31.2% | 26.2 to 36.3% | 27,620 |
Three points stand out.
The gap is large and the intervals are wide. Astra and GPT-6.1 Sol overlap within their confidence intervals, so the post does not support a claim that Astra beats Sol 6.1 with certainty. The jump from the GPT-5 family to the GPT-6 family is clearer.
Better models used fewer tokens. Astra used 4,704 output tokens per task. GPT-5.2 used 27,620 for a score of 31.2%. In this benchmark, more thinking text did not buy more accuracy. OpenAI also says Astra improves on GPT-5.6 Sol "even at low reasoning effort."
The confidence intervals come from 100 tasks. OpenAI resampled whole tasks and kept the five attempts for each task together. With only 100 tasks, a few tasks can move a score. Treat small gaps with care.
If you want the launch context for these models, see our coverage of the GPT-6 Astra launch and the GPT-6.1 Sol launch. OpenAI's other recent test of everyday work, LifeVal, covers childcare and paperwork rather than the bench.
Where did Astra improve on GPT-5.6 Sol?
OpenAI groups the gains into three examples. Each one shows a skill a lab scientist would call basic.
Knowing what the data cannot show
In one imaging task, GPT-5.6 Sol labeled cells as dead because a dead-cell dye lit up. But the detector could also pick up light from a second dye. The two detection windows overlap at 525 to 540 nm, so brightness alone could not justify the claim. Astra noticed the ambiguity and asked for better controls to separate the signals. OpenAI notes that the amount of interference "has not been measured."
Reading the small print on the plate
A DNA cloning task depended on a negative control plate. GPT-5.6 Sol saw no colonies on it and built its conclusion on that. Astra noticed the rim label read "LB Kan50", which means kanamycin, while the experiment used ampicillin. The plate was the wrong control, so Astra rejected it. This is the kind of detail that costs a real lab days.
Holding the experimental context
A microscope task showed cells appearing and vanishing across a series of images. GPT-5.6 Sol looked for a biological reason. Astra recognized a z-stack, a series of images taken at different depths. The "disappearing" cells were simply out of the focal plane.
Where does GPT-6 Astra still fail?
Gauge with a narrow green band, a picture of how far wet-lab AI agents are from reliable judgment
OpenAI says directly that "higher benchmark scores do not yet mean models can make reliable judgments in the lab." It then shows three failure cases.
| Failure | What happened | Why it matters |
|---|---|---|
| Assumption beats controls | In a colony PCR task, Astra assumed a larger PCR product meant successful cloning and proposed lanes 4, 6, 7 and 8. The reference selects lanes 5, 9 and 10 | It grouped the bands correctly but swapped what each group meant, so the wrong plasmid would be purified downstream |
| Contradicting data ignored | Astra said vacuoles are generally darker than the cell, though some in the same image are lighter. Focus changes can reverse contrast | It inspected the lighter vacuoles, commented on focus, then dropped them from the final answer |
| Counting errors | Astra's marks merged close colonies, missed visible ones and marked empty agar. Net result: an overcount of about 12% against the reference | An overcount inflates the estimate of how efficiently DNA entered the cells by the same proportion |
The second failure is the most telling. The model saw the contradiction and still chose the neater story. A human reviewer has to watch for exactly that pattern.
Why does this benchmark matter for builders?
Most science benchmarks test recall or reasoning on clean inputs. This one tests perception plus judgment on dirty inputs. That is the gap between "the model knows protocols" and "the model can be trusted to flag a bad control."
For teams building lab assistants or data-analysis agents, three takeaways follow.
- Check controls explicitly. Astra's wins and losses both turn on controls. Add a step to your agent's prompt that lists each control and states whether it is valid before any conclusion.
- Do not trust counts. A 12% overcount from a routine task is large enough to change a downstream estimate. Use a validated counting tool or human spot checks.
- Make the agent say "insufficient evidence." The rubric gives credit for recognizing that evidence cannot support a conclusion. Your own evaluations should too.
This is also a theme in other recent science coverage. Anthropic's report on Claude scientists research sessions, Google's jagged frontier in science report and OpenAI's Navier-Stokes agent swarm result all point to uneven skill: strong in some steps, weak in basic checks.
Anthropic published a related essay the same day, on how AI builds and interprets scientific models. We cover it in our analysis of that Anthropic guest post. It argues that a fitted answer is not understanding, which fits the pattern here: a model can get the grouping right and the meaning wrong.
What is claimed and what is not verified?
Everything above comes from OpenAI's own post. We have not seen an independent replication. Points to keep in mind:
- OpenAI graded its own models. No outside lab's models are in the table, so the post cannot tell you how a rival compares.
- The task set is small. 100 tasks across five domains means each domain has a modest number of tasks.
- No public release is described. We found no link to the tasks, rubrics or grading code in the post. Without them, outside groups cannot reproduce the scores.
- Biosafety is mentioned, not detailed. OpenAI says tracking these capabilities will help it understand "implications for biosafety and biosecurity." It gives no policy details.
- Ligo is a partner and not an independent auditor. Ligo scientists wrote and reviewed tasks. An external PhD-level scientist also reviewed each task, which helps.
What should you do next?
If you build in this area, read OpenAI's failure cases before the leaderboard. They double as a test plan for your own agent.
- Run your agent on a gel image where the ladder is missing and see whether it admits the gap.
- Feed it a plate with the wrong antibiotic label and check whether it rejects the control.
- Give it a z-stack and see if it calls out the depth change.
- Compare its colony count with a human count on 10 plates.
Bottom line
BioBasicsBench shows real progress: Astra's 66.7% is well above GPT-5.6 Sol's 43.8%, with fewer tokens. It also shows that a third of the available credit is still out of reach, and that the misses are the kind that waste experiments. OpenAI's honesty about those misses is the most useful part of the release.
Figures are from OpenAI's November 6, 2026 post and may change as OpenAI updates it. Check the source before you cite a number.
Related reading
- GPT-6 Astra launch: benchmarks and pricing
- GPT-6.1 Sol launch: pricing and benchmarks
- OpenAI LifeVal: everyday tasks benchmark
- Google's AI in science report and the jagged frontier
- Claude scientists: 207,000 research sessions
- Anthropic life sciences verification program
- Anthropic on teaching AI to build scientific models
- Anthropic research post (source)