A scientific software rewrite can correlate 0.991 with its reference implementation and still be wrong in ways that matter. That happened in a contributor-reported case study of an AI-assisted rewrite of bayesm.HART: 11 of 14 coefficient means exceeded the prespecified tolerance, one focal coefficient differed by -1.20, and the first implementation ran 3.6 times slower than the original.
This is the useful warning for national laboratories and research teams adopting AI agents. The main control problem is no longer whether an agent can produce code. It is whether the research workflow exposes the kinds of errors that a plausible aggregate score conceals.
The right unit of infrastructure is therefore a verification stack: external-reference checks, parameter-level tolerances, simulated ground truth, algorithmic diagnostics, and real-data scientific validation. No layer can borrow confidence from another. Passing ordinary software tests does not establish statistical validity, and matching one aggregate statistic does not establish scientific equivalence.
National-scale access makes verification the bottleneck
OpenAI's July 2026 national science commitments include Codex access for approximately 2,000 Genesis researchers, API support for large scientific campaigns, and work connecting models with national-laboratory supercomputers, simulations, facilities, and expert teams. The announcement also puts researchers at the center of choosing methods, challenging outputs, and validating results.
That emphasis is important. More model access increases implementation capacity immediately, but it does not automatically increase review capacity. An agent can migrate a package, optimize a kernel, add a sampler, and generate a benchmark before a domain expert has defined what evidence would make the result trustworthy.
OpenAI's accompanying scientific computing field report describes eight agent-assisted projects. Its recurring theme is that agents handled well-scoped engineering tasks effectively, while humans still had to decide whether outputs were scientifically valid. The strongest projects used external references or measurable acceptance targets, proceeded in stages, and spent substantial effort on the last mile of numerical and scientific validation.
The report also states an essential evidence boundary: project-specific benchmarks and validation results are contributor-reported unless otherwise noted. The report authors checked internal consistency and some public artifacts, but did not independently reproduce every result. The numbers below should be read within that boundary.
The 0.991 failure
The bayesm-rs project rewrote selected Bayesian models and samplers from the R package bayesm in Rust, then added two statistical extensions: an HMC/NUTS path and a nonlinear heterogeneity path derived from bayesm.HART.
For the initial HART comparison, the contributors used the bank-data example from the original documentation: 946 respondents, 14 coefficients, and 5,000 draws. The respondent-by-coefficient prediction surface from the rewrite had a correlation of 0.991 with the original. It also reproduced much of the focal segment separation.
Those numbers looked reassuring. More specific checks were not:
- 11 of 14 coefficient means differed by more than 0.25 reference standard deviations.
- The focal
Out_Statecoefficient had a signed difference of -1.20. - Runtime was 50.0 seconds versus 13.9 seconds for the original, and eight threads did not improve it.
- The rewrite used one fixed shrinkage value where the original adapted shrinkage to each coefficient's scale.
- A reused update rebuilt a large matrix every iteration, producing quadratic growth in the expanded design.
The separate HMC/NUTS extension also passed an initial population-mean agreement criterion despite an inverted diagonal mass matrix. Simulation-based calibration later exposed another trajectory-construction defect.
The lesson is not that correlation is useless. It is that every metric has a field of view. Correlation saw the broad shape of the prediction surface. It could not establish parameter fidelity, prior fidelity, sampler correctness, or computational scaling. A scientific acceptance gate must be composed from checks with different failure sensitivities.
The public HART pull request is still marked Draft at the time of writing and describes the backend as experimental rather than bit-for-bit equivalent to upstream HART. That is another reason to separate an instructive validation case from a claim of production readiness.
A five-layer verification stack
Layer 1: external-reference parity
Start with a trusted implementation, analytical result, curated fixture, or experimentally established answer. Freeze the exact reference version and compare at the narrowest meaningful interface.
For deterministic code, this may mean exact output agreement. For stochastic scientific software, it usually means a distributional or tolerance-based contract. The reference is not automatically truth, but it creates a stable comparison boundary and makes unintended behavior changes visible.
The acceptance specification should name:
- the dataset and preprocessing version;
- the reference implementation and commit;
- priors, initialization, seeds, and chain settings;
- the quantities being compared;
- the tolerance and the reason it is scientifically acceptable.
Layer 2: parameter and invariant tolerances
Aggregate agreement is a summary, not an interface contract. Decompose it into parameters, subgroups, conservation laws, boundary cases, or other domain invariants.
The HART case illustrates why. A 0.991 prediction-surface correlation coexisted with widespread coefficient-level gaps. Those gaps could alter interpretation even if top-line predictions remain similar. In physics, the analogous miss might violate a conservation law. In genomics, it might appear only for a variant class. In climate modeling, a global average can hide regional bias.
Set tolerances before seeing the candidate result. A threshold selected after inspection is a description of what happened, not an independent acceptance test.
Layer 3: simulated-data ground truth
When the system estimates latent quantities, real data often cannot reveal whether the inference procedure is calibrated. Simulation can.
Simulation-based calibration repeatedly samples parameters from the model's prior, generates synthetic data, fits the model, and records where the true parameter falls among posterior draws. A correct implementation should produce approximately uniform rank statistics under the tested generative model. The field report says the corrected samplers produced rank histograms consistent with uniformity across 500 to 1,000 replications per model.
This check finds errors that agreement on posterior means can miss. It does not prove that the model represents the physical world. It proves a narrower and valuable statement: under the simulated model and tested settings, the inference algorithm behaves consistently with the specified probability model.
Layer 4: algorithmic and numerical diagnostics
Scientific correctness includes how an answer was obtained. Samplers need convergence and effective-sample diagnostics. Optimizers need sensitivity and stability checks. Solvers need residuals, conditioning, and error bounds. Parallel implementations need scaling tests that isolate the dimension driving cost.
Keep correctness and performance gates separate. A faster wrong implementation has failed. A correct but unexpectedly superlinear implementation may also be unsuitable for real workloads. In the HART case, varying only the number of basis columns exposed the matrix rebuild as the runtime bottleneck. After correction, the report says the relevant update fell from 4.71 seconds to 0.56 seconds at 44 columns, and the rewrite became about 2.6 times faster than the original.
Layer 5: real-data and scientific validation
The final layer asks whether the software supports the intended scientific claim. This may require held-out prediction, replication on independent datasets, comparison with laboratory measurements, expert review of error modes, or a prospective experiment.
Real-data validation cannot replace simulation truth because the true latent parameters are unknown. Simulation truth cannot replace real-data validation because a perfectly implemented model can still be a poor model of nature. The two layers answer different questions.
For consequential work, define the release boundary explicitly. A tool may be valid for exploratory analysis but not for clinical, safety, regulatory, or national-security decisions. “Research use only” should correspond to a real control in the workflow, not a sentence in the README.
Freeze the experiment before asking the agent to optimize it
Verification becomes weak when the candidate can change the test, the reference, or the success threshold while changing the implementation. Freeze a small evidence bundle before the main run:
| Artifact | What it prevents |
|---|---|
| Input manifest and hashes | Silent changes to data or preprocessing |
| Environment and dependency lock | Results that cannot be reconstructed |
| Reference commit and command | Moving comparison targets |
| Acceptance specification | Post hoc threshold selection |
| Seed and run manifest | Untraceable stochastic variation |
| Raw results plus comparison script | Claims that exist only in prose |
An agent may propose improvements to the validation plan, but changing the plan should create a new version and invalidate the old comparison. This turns verification into an interface: inputs, outputs, tolerances, and evidence are inspectable by someone outside the generating context.
Use evidence states, not one “passed” label
A single green check encourages status inflation. A more honest release record distinguishes what has actually been established:
- Candidate: code exists and ordinary tests run.
- Reference-checked: named outputs meet frozen comparison tolerances.
- Simulation-calibrated: synthetic-truth tests pass over declared regimes.
- Domain-reviewed: experts have inspected scientific assumptions and failure modes.
- Independently reproduced: another team has reconstructed the result from the preserved artifacts.
- Stewarded release: an owner, regression suite, compatibility policy, and maintenance path exist.
These labels are cumulative only when their evidence is preserved. A draft pull request can be reference-checked without being independently reproduced. A field-report benchmark can be credible and useful while remaining contributor-reported. Precise labels improve trust because they state the boundary of the claim.
The same admission-control principle appears in our dual-loop verification protocol for mathematical agents: uncertain work can remain exploratory, but durable knowledge needs an explicit promotion gate.
A practical review protocol for scientific Agent work
Before approving an agent-generated scientific change, require a short verification record:
- Claim: What scientific or computational property is supposed to hold?
- Reference: What external result or prior implementation anchors the comparison?
- Acceptance targets: Which aggregate, parameter-level, invariant, diagnostic, and performance thresholds were frozen?
- Simulation regime: Which truths were generated, over what parameter range, and how many replications were run?
- Real-data boundary: What has been checked on observational or experimental data, and what remains unknown?
- Evidence status: Contributor-reported, artifact-backed, independently reproduced, or operationally monitored?
- Owner: Who maintains the code, reruns regressions, evaluates future model changes, and decides whether to release?
This protocol does not require every project to formalize everything. It requires each project to say which layers apply, which ones passed, and which ones remain open.
Verification capacity is scientific infrastructure
As national laboratories gain access to more capable agents, implementation throughput will rise faster than expert review unless verification is built into the environment. Shared datasets, reference containers, simulation harnesses, benchmark registries, artifact retention, and review ownership are therefore not administrative overhead. They are the infrastructure that converts cheap generated code into reusable scientific capacity.
The durable advantage will not come from one model producing the most code. It will come from institutions that make scientific errors easy to expose, evidence easy to reproduce, and responsibility impossible to lose.
FAQ
What is a verification stack for scientific AI agents?
It is a layered set of checks that evaluates an agent's output against external references, local parameter or invariant tolerances, simulated ground truth, numerical diagnostics, and real-world scientific evidence. Each layer targets different failure modes.
Why are unit tests not enough for scientific software?
Unit tests can verify expected code paths and known examples. They often cannot establish statistical calibration, numerical stability across regimes, fidelity to a reference method, or validity of the scientific assumptions.
Does simulation-based calibration prove a model is scientifically correct?
No. It tests whether an inference implementation recovers known truths under data generated from the specified model. A model can be internally calibrated and still represent the physical or biological system poorly.
When can an AI-generated scientific rewrite be considered production ready?
Only after its required verification layers pass and the project has an owner, reproducible artifacts, a regression suite, a release boundary, and a maintenance plan. A successful benchmark or draft pull request alone is insufficient.