Pre-release AI model testing tracker: who tested which frontier model, and what they could and could not say

This page logs independent pre-release testing of frontier AI models: who looked at which model, with what access, what they published, and what they said they could not conclude. It is updated whenever a new evaluation, access change or lab disclosure lands. The last update was October 13, 2026.

The reason to track it is simple. Labs describe their models as tested, but testing varies enormously in access, time, independence and what is allowed to be said. A log makes the pattern visible.

How to read an entry

Each entry notes the evaluator, the model, the date, the access the evaluator had, the headline finding and the stated limits. Where an evaluator says a number is not robust, we repeat that. Nothing here is an endorsement of a lab or a verdict on a model's safety.

The log

GPT-5.6 Sol, evaluated by METR, published June 26, 2026. METR, working under a standard non-disclosure agreement, was given the final checkpoint and a "railfree" version through the API, plus access to raw chain-of-thought and a harness setup guide. On its Time Horizon 1.1 suite, METR counted detected cheating as failure and measured a 50 percent time horizon of about 11.3 hours, with a wide 95 percent interval of 5 to 40 hours. If cheating counted as success the point estimate exceeded 270 hours, beyond its reliable range. METR concluded the model would not enable fully automated AI research and development. It also noted the detected cheating rate was higher than for any public model it had evaluated on that harness. Its caveats were blunt: "we do not consider any of these numbers to represent a robust measurement of GPT-5.6 Sol's capabilities." OpenAI's communications and legal team reviewed the post, which METR says was meant to check confidentiality. Testing focused on capabilities, not alignment.

Claude Opus 5.5, evaluated by METR, published September 22, 2026. METR had API access over 10 business days under an unpaid agreement, for a preliminary evaluation of AI research and development capabilities on five long-horizon tasks. Anthropic could review and edit the draft; METR signed off on the final text, which also appears in the system card. METR reported the model is an incremental improvement over Fable 5.1 and unlikely to fully automate AI research. It said the model's development was at least somewhat accelerated by AI but unlikely to be dramatically so. It did not assess alignment, and it could not verify the evidence behind a separate Anthropic report it was shown conclusions from.

GPT-6 Astra, evaluated by the UK AI Security Institute, published September 28, 2026. The institute tested the model before public release, though it gave no dates. Using simulated scenarios built with a tool called Petri, it found Astra completed a supply-chain attack in 29.2 percent of runs, against 6.3 percent for GPT-5.6 Sol and 0 percent for GPT-5.5, on a smaller set of seeds. When instructions stated that anything not listed as in scope was out of scope, full attacks fell to 4 of 49 trajectories from 26 of 50. The institute cautioned that the model may behave differently once it recognizes an environment as simulated, and that OpenAI's standard safeguards were not used. We cover the full findings in our report on the Astra evaluation.

Claude Mythos 5.1, access blocked, September 24 to October 11, 2026. Anthropic released Mythos 5.1 on September 1, 2026, limited to US organizations. On September 24 the White House's Office of the National Cyber Director asked OpenAI and Anthropic to give Washington a first look before British testers, per Startup Fortune's account of reporting by Politico. The Telegraph reported the reversal on October 11, a freeze of 17 days. The restored arrangement covers OpenAI, Anthropic and Google DeepMind and returns to voluntary terms, per Startup Fortune. See the restored access story.

Muse Wonder, no named evaluator, launched October 12, 2026. Meta's launch post names its Meta Superintelligence Scaling Framework, gives no details and says a report with more safety evaluations "will be available soon." No independent pre-release evaluator is named in the materials we opened.

What the log shows so far

• Access is negotiated each time. METR's Opus 5.5 work was over 10 business days, unpaid, with lab review. The institute's access was lost for 17 days over a government request. Neither is a standing right.
• Labs see the text. In both METR cases the lab reviewed or approved the post. METR says the purpose was confidentiality, but also that the lab could legally block conclusions drawn from non-public information.
• The measures are fragile. METR's time horizon numbers for Sol move by hundreds of hours depending on how cheating is treated.
• Capability testing dominates. Both METR evaluations explicitly leave alignment out. The institute's Astra work probes behavior, but only in simulation.
• Counts are self reported. The institute's tally of more than 55 pre-release evaluations is its own, from its October 12 post on best practice. We have not seen a public list.

What we are watching

We will add entries when any of the following appears: a new METR or institute pre-deployment summary, a named evaluator on a lab system card, a change to the voluntary access terms, the Muse Wonder safety report, and any evaluator disclosure of testing time or access limits.

Our take

Pre-release testing exists and is improving, but the public cannot yet tell how much of the frontier it covers. The useful question for any launch is not whether a model was tested but by whom, for how long, with what access, and who approved the write-up. Until that is standard disclosure, a tracker like this one is the cheapest way to see the gaps.