Can AI agents rebuild biological AI tools from a paper? AISI's ReproBAIT says partly, for under $20

Can AI agents rebuild biological AI tools from a paper? On October 26, 2026 the UK AI Security Institute (AISI) published its answer: partly, cheaply, and well enough to undercut a common safeguard. In an evaluation called ReproBAIT, built with SecureBio, four coding agents rebuilt selected functions of published protein and molecule design tools in a sandbox with one GPU and no internet access. The simplest rebuilds took under four hours and under $20.

What AISI tested

The institute's chem-bio team asked a narrow question: if the code for a biological AI tool (a BAIT) is withheld and only the paper is public, can a coding agent reproduce the tool anyway? Managed access to such tools often rests on exactly that assumption.

Four agents were tested, two closed and two open-weight: Claude Opus 5, GPT-5.6 Sol, Kimi K3 and Qwen3.8-2.4T. Three target tools were chosen. ProteinMPNN and RFdiffusion3 are open-source protein design tools, with RFdiffusion3 the more complex. CMD1 is a closed-source molecular design tool.

Each agent worked under two conditions: abstract only, or the full paper with supplementary material. For the open-source tools there was also a baseline with the original code. Every attempt ran in an isolated sandbox with one GPU and no internet access, with a limited description and the training data. A separate, network-isolated scorer judged the output on held-out data. The metric, reproduction fidelity, is the rebuilt tool's score as a percentage of the original's on the same measure, where 100 percent means parity. Each configuration ran three times and the best attempt was reported.

What came back

Results split by tool, and the split is the story.

• CMD1: all four agents matched or exceeded the original on the validity metric. AISI cautions that this metric was already near saturation, and that agents optimized for it in ways that slightly reduced molecular novelty.
• ProteinMPNN: the strongest agents approached the original on sequence recovery. Claude Opus 5 reached 96.5 percent fidelity with the full paper. Results varied across agents.
• RFdiffusion3: no agent produced a successful reproduction within its time budget. The one partially successful agent narrowed its training data, and further runs were excluded because they returned near-duplicates of known Protein Data Bank entries.

Cost tracks difficulty. The two less complex tools were rebuilt for roughly $4 to $20, and the simplest within four hours. The failed RFdiffusion3 attempts cost $102 to $301 each, so failure was not cheap either.

Two findings that cut against intuition

First, giving the agents the full paper did not consistently beat giving them only the abstract. That matters for a proposed control: limiting how much a paper discloses did not show a clear protective benefit in this test.

Second, memorization does not appear to explain the results. AISI scanned transcripts for recalled details and compared agent code with the originals for verbatim matches. It reports moderate evidence of memorization for ProteinMPNN and little for RFdiffusion3, and it calls these diagnostics imperfect and directional only. That is a fair caveat, and it means the safest reading is that memorization is a partial explanation for the easy case and a minor one for the hard case.

What AISI concludes, and what it does not

The authors do not claim agents can fully reproduce the originals. They say selected functions can be rebuilt with limited time and compute, and that the trend points toward more. Their conclusion on access control is direct: "withholding code does not, by itself, assure control over access to a BAIT's capability." They propose a different lever: "controlling access to the underpinning biological data appears to be a more promising intervention."

The data-first argument has independent support. A 2025 arXiv paper, "Deep Ignorance," found that filtering dual-use topics out of pretraining data produced models that resisted adversarial fine-tuning for up to 10,000 steps and 300 million tokens of biothreat-related text, with no observed loss in unrelated capabilities. AISI also published a companion study the same day on filtering protein-model training data. Two separate lines of work now point at data, not code.

Limits of the evaluation

The scope is small: three tools, four agents, three runs each with the best attempt reported. Best-of-three flatters the agents relative to a typical run, and the RFdiffusion3 failures were judged within each agent's time budget, so more time might change that result. The one tool that failed, RFdiffusion3, is the most complex, so the headline for a defender may be that complexity buys time, not that code secrecy does.

There is a second, quieter caveat. AISI tested both closed and open-weight agents, and open-weight models are by definition outside any provider's access policy. That is our inference, not AISI's, but it fits the institute's turn toward the data.

Why this matters for policy

UK government has put money behind biosecurity testing. Our coverage of the UK AI biosecurity lab announcement and of how AISI and the US institute CAISI divide testing work gives the institutional background. ReproBAIT is the kind of result that tells a regulator where to aim: at who can obtain biological training data, and under what controls, rather than at which papers get published.

Our take

Treat this as a measured, bounded result with a clear direction. Cheap rebuilds of simple tools are real; a full reproduction of a complex tool is not shown. If you set or follow policy on biological AI tools, the finding to carry forward is that code secrecy alone is not a control, and that data access is the next place to look. What we would watch: repeat runs with more tools, a harder target than RFdiffusion3 that does get rebuilt, and any government response that names data access controls explicitly.