Open-weight AI cyber capability gap tracker: how far behind is the best open model?

How far behind are open-weight models at cyber tasks? As of October 2026 the best available answer is a range, not a number: about four months by NIST's Center for AI Standards and Innovation (CAISI), and six to seven months by Epoch AI. The difference comes from method, which is why this tracker logs both.

This page follows one question over time. It will be updated, not replaced, when a new open model, evaluator or index revision arrives. Every row names who measured, when, and with what.

The current readings

Epoch AI, October 16, 2026. Its Cyber ECI is a cybersecurity-focused version of the Epoch Capabilities Index. Z.ai's GLM-5.3 has the highest open-weight score at 160.10. Epoch says the open cyber frontier is "roughly where the closed frontier stood 6 to 7 months ago." Comparison points: GPT-5.4 (released March 5, 2026) scores 157.04, below GLM-5.3, while Claude Mythos Preview (April 7, 2026 checkpoint) scores 164.82, above it. At the top, Claude Opus 5.5 and GPT-6 Astra both score 171.

NIST CAISI, September 17, 2026. CAISI calls GLM-5.3 "the most cyber-capable open-weight model released to date" and says it "lags the capability level of the U.S. frontier by about four months" in an aggregate across its cyber benchmarks. Z.ai released the model on August 14 and published its weights two weeks later.

Anthropic, September 29, 2026. Anthropic's own tests found GLM-5.3 built end-to-end exploits in 50 of 410 ExploitBench attempts on the V8 engine, against 56 of 410 for Claude Mythos Preview. On its internal binary exploitation benchmark of 100 random tasks, GLM-5.3 achieved full control-flow hijacks in 4% of trials, against 6% for Mythos Preview.

Why four months and six to seven months can both be right

Different yardsticks. CAISI's number is an aggregate over its benchmarks, and its own page does not explain how the index was converted to months. Epoch's estimate places the open model on a timeline of closed releases using a joint fit across general and cyber benchmarks.

Epoch is explicit about its weak points. GLM-5.3's score rests on only three CAISI results, and Opus 5.5's on four Anthropic-reported benchmarks. CAISI did not name the US models it evaluated, so Epoch attributed results to the models it judged most likely, such as GPT-6 Astra and Claude Mythos Preview. An October refit lowered several top scores, including Claude Mythos 5 by 3.9 points and GPT-5.6 Sol by 4.7, so earlier Cyber ECI values are not directly comparable. Data is current to October 8, and the October update expanded the index from 56 to 76 models and from 20 to 35 cyber benchmarks.

CAISI also notes that its comparison excludes models that are developed but not yet released. The closed frontier may be further ahead than any public index shows.

The benchmark detail

CAISI reported these results for GLM-5.3, with the US frontier best in parentheses: SEC-Bench Pro 40.4% (90.2%), ExploitBench 61.1% (100%), ExploitGym userspace 9.4% (44.4%) and CAISI OSS-Fuzz 7.7% (23.2%). On every row there is a large gap. The averages hide that the open model is close on some tasks and far on others, which is why a single month count is a summary, not a finding. We unpack the original dispute in our report on GLM-5.3's cyber capabilities.

The safeguards column

Capability is half of the risk. Anthropic reported that attackers could bypass GLM-5.3's safeguards 64% to 100% of the time in its simulated tests. Engagement with overtly harmful cyber orders rose from 0% on direct requests to 64% with a deceptive red-team prompt, 92% with prefilled reasoning and 100% once the model was abliterated.

Anthropic said abliteration took about 2,200 GPU hours, roughly $4,400, though it estimates an experienced team would need about 600 hours, or $1,200. It argues that a "critical threshold in freely accessible capabilities has now been crossed" and calls for government safety testing of capable models. That is a position from an interested lab, and it sits next to its own commercial interest in closed models.

How to read the gap

A lag of months matters because defenders and attackers both get access to the open model on release. It also matters less than it looks if the closed frontier is moving fast: Epoch's 6 to 7 months is a distance in time, not a fixed capability difference, and it can widen or shrink with each release.

What would change this page

We update on any of: a new open-weight release that tops 160 on Cyber ECI, a new CAISI or UK AI Security Institute assessment, a revision of Epoch's index, or a lab publishing results for an unreleased closed model. Each entry will carry its date.

Method notes for anyone updating this page

Keep three rules when adding a row. Record the evaluator, the date of the data and the model checkpoint, because Epoch's October refit shows how scores shift when an index is rebuilt. Separate vendor-run tests from independent ones: Anthropic's results are useful, but it is both an evaluator and a competitor to open models. And record sample sizes, since a score resting on three benchmark results, as GLM-5.3's does in Epoch's index, is a weaker claim than one resting on thirty.

Our take

The two estimates agree on direction and disagree on size, and that is a reasonable place to be. Our working range is four to seven months as of October 2026, with the larger figure better documented. The uncomfortable fact is not the gap. It is that an open model near the closed frontier can have its refusals removed for a few thousand dollars. Treat the lag as how long the closed labs have before the same capability is freely downloadable.