FreshSynapse

AI trends & daily updates, in plain English

AI Research

OpenAI's New Test Asks AI to Read Real Lab Evidence — and Its Best Model Scores 66.7%

By FreshSynapse Editorial · · 3 min read

A scientist in a white coat and yellow gloves using a pipette to add purple liquid to a flask in a laboratory

Photo: Hippopx (public domain). Illustrative image.

What happened. OpenAI introduced a new benchmark on Thursday 6 November 2026 that tests artificial intelligence agents on the evidence scientists actually produce in wet laboratories — the messy, ambiguous material of everyday bench work, rather than the clean charts selected for published papers. BioBasicsBench was built with Ligo, a biotechnology company focused on enzyme design, and contains 100 tasks drawn from real experiments.

What it measures. The tasks span five areas of routine laboratory work: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. Agents must interpret gel and blot images, colony plates, microscopy stacks, sequencing reads, plate-reader exports, bench photographs and protocol videos — and recognise when the evidence cannot support a conclusion. Scientists built the tasks from their own experiments, each paired with an expert-written grading rubric. Every task was reviewed internally by a Ligo scientist who had not written it, externally by a PhD-level wet-lab scientist, and once more internally.

The results. OpenAI evaluated seven of its own models across 37 combinations of model and reasoning effort, with five attempts on every task — 18,500 scored responses in total. Scores are reported as mean reward, the average share of available rubric credit, including partial credit: they are not the share of tasks fully solved. At the “xhigh” reasoning setting, the highest reported for all seven models, GPT-6 Astra earned 66.7% of the available credit (95% confidence interval 60.3–73.1%), ahead of GPT-6.1 Sol at 62.0% and GPT-5.6 Sol at 43.8%. Astra also used about half the output tokens of its predecessor per task — 4,704 against 9,099.

Where Astra did better. OpenAI highlights three kinds of improvement. In a cell-imaging task, GPT-5.6 Sol classified cells as dead from a dye signal; Astra recognised that overlapping detection windows could let a second dye interfere, and called for controls that could separate the signals. In a DNA-cloning task, Sol based its conclusions on seeing no colonies on a negative-control plate; Astra noticed the plate was labelled for kanamycin rather than the ampicillin used in the experiment, and rejected it as a reliable control. And where Sol sought a biological explanation for cells apparently vanishing in a series of microscope images, Astra correctly recognised a z-stack — images taken at different depths.

Where it still falls short. OpenAI is careful with the framing: “Higher benchmark scores do not yet mean models can make reliable judgments in the lab.” In a colony-PCR task, Astra assumed a larger PCR product indicated successful cloning and selected lanes 4, 6, 7 and 8; the reference interpretation selected lanes 5, 9 and 10 — a mistake that would lead to purifying the wrong plasmid. In another task it generalised that vacuoles are generally darker than surrounding cell material despite lighter vacuoles in the same image, and in a colony-counting task it overcounted by about 12%, which would inflate the estimated efficiency of DNA introduction by the same proportion. The pattern OpenAI draws out is that a correct intermediate observation can still fail to govern the final answer.

The limits of the test. All seven models tested are OpenAI’s own, so the leaderboard says nothing about rival systems. No human baseline was published, and the dataset and rubrics were not publicly linked at launch. OpenAI says tracking these capabilities will also inform its assessment of their biosafety and biosecurity consequences — a benchmark that makes models better at laboratory work also makes them more capable in a domain with dual-use risks.

Sources

  • OpenAI
  • GPT-6 Astra
  • BioBasicsBench
  • benchmarks
  • biology
  • AI research

Related articles