Verifiable Environments for AI in Biology — Kenny Workman, LatchBio
Biology agents need evaluators that verify analysis of large experimental datasets, not recall. LatchBio found human review essential because valid scientific paths can defeat brittle graders.
A single-cell run can produce **2–6 TB**, while a spatial biology run can reach **7 TB**. LatchBio’s Spatial Bench contains **146 problems** with data inputs, scientific tasks, grader configuration, and deterministic checks.
Builders of research agents should require conclusions to come from interacting with the supplied data. Ground truth must remain valid across legitimate analysis paths, and human attempts should test whether deterministic graders reject scientifically sound alternatives.
A single-cell run can produce **2–6 TB**, while a spatial biology run can reach **7 TB**. LatchBio’s Spatial Bench contains **146 problems** with data inputs, scientific tasks, grader configuration, and deterministic checks. Builders of research agents should require conclusions to come from interacting with the supplied data. Ground truth must remain valid across legitimate analysis paths, and human attempts should test whether deterministic graders reject scientifically sound alternatives. End-state rewards become weak as workflows grow longer, and current models still miss full biological tasks. Each long-horizon evaluation reportedly took **three people about a week** to create, showing how expensive durable domain verification can be.
This makes benchmark integrity concrete for data-intensive biology: deterministic checks must verify conclusions derived from supplied data without rejecting scientifically valid alternative analyses. It reinforces final-state verification while narrowing its applicability—long biological workflows weaken end-state rewards, current models remain incomplete, and durable expert-built tasks are costly to produce.