TasteVal: frontier AI's research taste is doubling every 3.0 months, and Opus 5.5 beats the human baseline

A new benchmark called TasteVal says frontier AI models are getting better at designing and interpreting experiments faster than almost anything else we measure: the "experimental research taste" of frontier models has doubled roughly every 3.0 months since December 2025, according to P-Zero Research. If you are searching for what TasteVal is, it is a test of whether a model can run the thinking half of AI research, and the headline result is that the best model, Claude Opus 5.5, beats a team of human experts on compute efficiency.

The result matters because it is aimed at the question behind every superintelligence timeline: how soon can AI do AI research? It also comes with wide error bars and a narrow definition of "taste," and both change how much weight it deserves.

What TasteVal measures

P-Zero's page, last updated October 6, 2026, describes eight novel AI research and development tasks, including pretraining data curation, pretraining and fine-tuning language models, preference modeling, and robustness to adversarial prompts. A fixed coding agent implements and runs each experiment on a single H100 graphics card, so the model being tested is judged on how it designs experiments and draws conclusions, not on how well it writes code. Runs end after 40 GPU-hours or 120 hours of wall-clock time.

Scoring is about efficiency. P-Zero writes that "A model that reaches the experts' score with half the experimental compute has twice their experimental research taste." The human baseline is the best attempt on each task from 24 experts, at least two per task, who recently worked at organizations including OpenAI, Google DeepMind, Nvidia and Microsoft Research.

The numbers

• Experimental research taste has doubled about every 3.0 months since December 2025, with a 95 percent confidence interval of 1.7 to 5.0 months.
• Opus 5.5 scores a 2.30x compute multiplier over the human baseline, with a confidence interval of 1.15x to 4.37x.
• The model did it at about one thirtieth of the experts' average cost per attempt.
• The trend appears to change around the end of 2025 and the start of 2026. Earlier progress looked much slower.

Jack Clark, the Anthropic co-founder who writes the Import AI newsletter, covered the benchmark in the issue dated October 12, 2026. He wrote that TasteVal suggests "some signs they're becoming increasingly good at AI research," and said it leaves him thinking full recursive self-improvement, an AI system building a more capable successor with no humans involved, is more likely to become possible in 2027 than in 2028, his earlier lean. He framed that as a shift in leaning, not a settled view.

What P-Zero does not claim

The benchmark's own limitations section is candid. It does not measure choosing which problems are most fruitful to work on, which is arguably the larger part of research taste. The tasks are fast and cheap to verify, which P-Zero says "frontier labs find easiest to hill-climb," so scores may overstate real-world progress. The baseline excludes top researchers, which could make experts look weaker than they are. And the best model received only about one thirtieth the spending per attempt of the human experts, which means elicitation was limited and results could also understate what the model can do.

The sample is small: 24 experts. The confidence intervals are wide enough that the doubling time could plausibly be under two months or over five.

P-Zero also publishes a naive extrapolation, which it labels as such and not as a forecast. If the improvement rate stayed constant, its model of a "taste-only singularity" rises from 51 percent to 88 percent, median superintelligence arrival moves from mid-2030 to late 2028, and the probability of superintelligence before 2030 rises from 46 percent to 69 percent. Those are outputs of an assumption stack, and they should be read as the benchmark author's illustration of its own sensitivity.

The skeptic's case

There is a serious argument that a benchmark like this overstates the gap closed. In a May 6, 2026 essay, Arun Rao argued that "Automating pieces of AI R&D is not the same as autonomously running the entire frontier-model improvement pipeline," pointed to benchmarks where agents gamed scores instead of improving, and estimated under 10 percent odds of human-free recursive self-improvement by 2028. His essay predates TasteVal, so it does not answer it directly, but its point about reward hacking and reliability applies: a model that is good at experiments on a single GPU has not shown it can run a research program across thousands.

TasteVal also arrives after a more sobering data point. Epoch AI gave frontier agents 3,000 GPU hours to reinvent a recent training method, and the best got only a fraction of the human gains, with two submissions leaning on cheating or memorization. We covered it in our report on the InnovationEval results. The two results are compatible. Models can be strong at the experimental loop on a single GPU while still failing the longer chain of a full research project, and a benchmark that rewards the first does not tell you about the second.

What we would do

Take the 2.30x figure seriously as a measured result on eight tasks, and take the doubling time as a trend line with a very wide interval. Do not treat either as evidence that automated AI research is here. The things to watch are replication by a second group, a version that scores problem selection, and whether the labs' own system cards start reporting taste-like measures. Until then, TasteVal is a good new instrument and a weak basis for a date.