FreshSynapse

AI trends & daily updates, in plain English

AI Research

AI Is Now Speeding Up Anthropic's Own AI Research by About 1.9x, Independent Audit Finds — but Most of the Evidence Is Redacted

By FreshSynapse Editorial · · 4 min read

A female scientist in a lab coat, hairnet and face mask working at a computer in a clean laboratory, with a microscope on the bench beside her

Photo: Hippopx (public domain). Illustrative image.

What happened. METR, the nonprofit that evaluates AI systems, published an independent assessment on Thursday 5 November of whether AI is accelerating research into better AI at Anthropic. Its headline finding: as of October 2026, Anthropic is “likely improving the capabilities of their models on well-specified tasks at around twice the rate they did between March 2024 and October 2025.” The assessment ran for 11 weeks, from 12 August to 29 October, with METR staff working on Anthropic devices with access to internal data on pretraining, reinforcement learning, compute and internal benchmarks, plus about 15 staff interviews. METR took no payment from Anthropic, which provided laptops and free tokens.

The 1.9x figure, and what it means. METR’s main measure was AECI, Anthropic’s version of the Epoch Capabilities Index, which rolls benchmark results into a single score. Since October 2025, METR estimates AECI has grown at 1.9 times its earlier rate, with a 95% confidence interval of 1.5 to 2.8 times. In plain terms, METR says that amounts to roughly two years of historical progress over the last year — about the distance from Claude 2.1 to Claude Sonnet 4.5. The underlying trend rose from about 15 AECI points a year before October 2025 to about 29 points a year after it. METR stresses the estimate is highly uncertain and says it “would not be surprised if the true acceleration was closer to 1.3x or 3x the historical rate.” Its robustness checks under alternative modelling choices produced estimates from 1.1x to 2.9x.

Is AI itself the cause? METR’s subjective estimate is that a majority of the acceleration as of October 2026 — perhaps about 80% — is due to AI, and it says the gains came mainly through assistive use of AI in research, not fully autonomous processes. It also says it is confident the jump was not caused by faster growth in training compute. But the 80% figure is a judgement, not a measurement, and METR says it carries substantial uncertainty. The distinction matters: faster capability gains and AI-caused gains are two separate claims, and the evidence for the second is thinner than for the first.

The redactions. This is where the report’s authority weakens. Anthropic had unilateral redaction rights over non-public information, and METR says the company “redacted a majority of the evidence we found most informative for these conclusions” — including per-stage acceleration and attribution estimates, the breakdown of compute, data and algorithms, bottleneck analyses, forecasts for future models and the specific benchmark names. Anthropic described the redactions as aimed at removing confidential information that could accelerate other AI developers. The result is that outside readers have to take METR’s word for conclusions they cannot check against the underlying data. The report’s figures also rest on internal-only models: readers of the report noted an unreleased “Model 4”, said to have finished training in October 2026 with an AECI score of 178.6, above Opus 5.5, and the first model in the analysis to score above 50% on every benchmark tracked. Anthropic has not announced such a model.

The safety-threshold question. METR also published a companion review of the automated research-and-development section of Anthropic’s August 2026 risk report. It agrees the risk from models as of 15 July was low, but says Anthropic’s claim not to have crossed its “2x Acceleration Threshold” — an early warning in its safety policy — rests on “contestable judgment calls”, that it is “plausible” the threshold had been crossed, and that the public report “overall understates the level of acceleration that was occurring.” Whether the threshold was crossed is a separate question from whether capabilities accelerated, and the two use different baselines and dates, so the two documents should not be read as contradicting each other directly.

Why it matters. The assessment is the clearest evidence yet for recursive self-improvement — AI systems helping to build more powerful versions of themselves — and it arrives as the industry debates whether such advances could outpace efforts to assess their risks. METR found no clear evidence of near-term bottlenecks in algorithmic progress, training compute, experiment compute or data that would necessarily slow further development over the next 12 months. That combination — faster progress, largely AI-driven, with no obvious brake — is the finding that will be quoted, and it is also the one that rests most heavily on evidence the public cannot see.

Sources

  • METR
  • Anthropic
  • AI research
  • recursive self-improvement
  • AECI
  • AI safety

Related articles