GDPx benchmark: SemiAnalysis says the best AI model scores 12 percent on high-value agent work

A new benchmark says the best AI model today completes only about 12 percent of a hard, realistic set of agent tasks. That figure, from SemiAnalysis, is less interesting than how the test is built and who wins which domain.

What GDPx is

SemiAnalysis, the semiconductor and AI research firm, published GDPx on November 2, 2026. The authors are Max Kan, Andrew Megalaa and Dylan Patel. They describe it as a test of general agentic capability in the areas where frontier models are likely to create the most value, arguing that "the first trillion dollars of AI revenue will be concentrated in sectors that can deploy frontier intelligence at high ROI." They deliberately leave out industries such as government, real estate and insurance.

It weights 11 domains equally: chip design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement, hardware design, materials science, alignment and software engineering. Tasks come from 14 specialized data partners, each subject to automated quality checks. They are interactive environments, not single prompts. A finance task, for example, means working through a deal room of roughly 2,000 files.

The default limit is 8 hours per task. Models run in their native harness where one exists, otherwise in OpenCode. The headline score is pass@1 across three attempts. Grading mixes binary and partial credit: biology is graded deterministically, and some domains use LLM judges working from expert rubrics.

The results

SemiAnalysis says the best model scores just 12 percent overall. The per-model overall numbers sit in charts we could not read as text, so we do not repeat them. The domain winners are described in words:

• Opus 5.5 is best all-around and wins quant trading, hardware design and chip design.
• Gemini 4 Carbon is the best biology model and edges software engineering.
• GPT-6 Astra wins recursive self-improvement and materials science.
• Muse Wonder Full is the best cyber model "by a meaningful margin," in the authors' words.

The authors note that Opus 5.5 and GPT-6 Astra score lower overall because of safety classifiers, and they publish a second ranking without cyber, biology and alignment. That is a real caveat: the same classifiers that keep a model from helping with dangerous tasks also depress its score on tests of those tasks.

Binary grading produced near-zero scores in finance and legal, so the authors report partial credit instead. In finance, on 11 checks, GPT 6.1 Sol averaged 5.6, Opus 5.5 and GPT-6 Astra 5.4, and GLM 5.3 and Grok 4.7 each averaged under 2. In legal, on one public task, Opus 5.5 averaged 44 percent, GPT-6 Astra 25 percent, Gemini 4 Argon 2 percent and GLM-5.3 0.5 percent. In chip design, on the Ergodex task, GPT-6 Astra averaged 0.34, Claude Opus 0.135 and Fable 0.07. In a hardware CAD housing task, none of 12 rollouts achieved a verified full pass.

How it relates to GDPval

GDPx is explicitly a reply to OpenAI's GDPval. The GDPval paper, posted to arXiv on October 5, 2025, covers 44 occupations across the top nine sectors by U.S. GDP, with a 220-task open gold subset, and concludes that "the current best frontier models are approaching industry experts in deliverable quality."

SemiAnalysis criticizes that design, saying it relies on spoon-fed context, noisy rubrics (71 percent inter-rater agreement on the gold subset, by SemiAnalysis's figure), dated tasks and no realistic environments. Artificial Analysis runs a variant, GDPval-AA, which scores models by Elo through blind pairwise comparisons on the same 220 tasks. Its latest page lists Claude Opus 5.5 at 1866, Kimi K3.1 Pro at 1858 and Muse Wonder Full at 1857, within each other's margins of error, though the page carries no date. See our benchmark saturation for why such gaps are small.

The two benchmarks disagree in spirit. GDPval says models approach experts at deliverables. GDPx says that, in long, messy environments, even the best fails most tasks. Both can be true, because they measure different things: producing a document versus completing a multi-hour job with real tools.

What to be careful about

Four cautions apply. SemiAnalysis sells related tokenomics models, and it built GDPx with its partners. Some domains rely on LLM judges. Several results use partial credit, which can make a model look better or worse depending on the rubric. And the examples the authors cite come from them and their partners, which we could not independently verify.

The two domains that touch the race itself deserve a note. Recursive self-improvement and alignment are on the list, so the benchmark measures, in part, how well agents can automate AI research and how they behave under oversight. Those are exactly the tasks safety bodies watch, and a benchmark that scores them gives outsiders a number to argue about.

What the domain split tells you

The most useful lesson is that no lab owns every row. If you are choosing a model for chip design or quant trading, the table points one way, and for biology or cyber it points another. That is consistent with what builders already report: the best model for your task is an empirical question, and an overall leaderboard rank hides the difference. It also means a company claiming its model is best at everything should be asked which of the 11 rows it would put in writing.

Our take

GDPx is a useful addition, mostly because it shows the spread: different labs lead in different jobs, and the best model's overall score is low. That cuts against both "AI replaces everything next quarter" and "benchmarks are saturated."

But it is one firm's test, and a weak claim until someone else runs it. We would watch for released task sets, for independent reruns, and for whether labs respond with their own numbers. Until then, the 12 percent is a data point about this test, not a measure of the economy.