METR estimates AI has roughly doubled Anthropic's research pace—how firm is that result?
The embedded evaluator found a 1.9-fold acceleration trend in an internal capability index, but Anthropic redacted substantial evidence and the report cannot isolate AI's contribution with experimental certainty.
The short version
- What happened
- Six METR employees now have Anthropic hardware, office space and access to internal tools and systems broadly comparable with the company's risk-assessment teams.
- Who it affects
- Workers, customers and communities cannot inspect frontier systems themselves; they benefit only if outside assessors can surface material risks clearly enough for companies and public authorities to act.
- What remains unproven
- METR has now published a completed embedded assessment, but it is an observational synthesis rather than a randomized experiment and cannot isolate AI's causal contribution with experimental certainty.
Keep in mind
METR has now published a completed embedded assessment, but it is an observational synthesis rather than a randomized experiment and cannot isolate AI's causal contribution with experimental certainty.
Anthropic redacted substantial supporting evidence, preventing full public reproduction of the internal index, benchmark analyses and attribution judgement.
The capability index and staff reports concern Anthropic's internal AI R&D process; they do not directly measure product safety, social benefit, scientific breakthroughs or effects at other organisations.
Multi-source analysis · 3 linked sources. Primary analysed source date: 15 October 2026. Latest linked supporting source: 5 November 2026.
Read the full report · 7 min The evidence, methods, implications and limitations
The access is unusually deep, but not unlimited
Anthropic says six employees of the nonprofit evaluator METR are now working with hardware, office space and internal tools roughly comparable with those used by its own risk-assessment teams. The embedded group may study model risk levels, incidents and company practices. Customer data, attorney-client privileged material and model architecture remain outside the arrangement. Anthropic says it cannot veto a METR publication, although it may require redactions within agreed categories and retains the final decision over classification. METR can say when those limits affected its access or reporting.[1][2]
Why the publication terms matter
For researchers, policymakers and people affected by frontier systems, the value is not proximity alone: it is whether evaluators can disclose consequential failures and unresolved disagreements. METR says it accepts no compensation for model-evaluation work, while developers provide model access and inference tokens. That reduces one financial conflict but cannot remove operational dependence on Anthropic's systems, security decisions and continuing cooperation. The arrangement is stronger than a narrow pre-release test, yet still voluntary and controlled by a company being assessed.[1][2]
The distinction that matters
Where evaluator independence is gained—and constrained
The agreement expands what METR can inspect, while reserving defined controls for Anthropic.
Access
Six METR staff use internal systems and workspace
More evidence than an external API-only test
Publish
METR writes its own assessments without an Anthropic veto
Findings can be released after required review
Limits
Anthropic excludes some data and makes final classification decisions
Redactions and access gaps still shape the public record
What the announcement establishes—and what it does not
Anthropic presents the METR arrangement as its highest of three evaluator-access tiers. The lower tiers include scheduled access for organisations such as SecureBio, Accenture and Faculty, and more limited evaluations by groups with their own methods. That structure may help outside specialists follow a model through development rather than testing a single release, but the announcement supplies no completed METR report, disclosure log or example showing how a contested redaction was resolved.
The company updated its announcement on 16 October to clarify the terms, one day after publication. The original 15 October date remains the source date. A credible test will be whether METR publishes specific findings, names important access gaps, separates company claims from its own judgements and can continue reporting after a critical result. External replication and regulator access would add safeguards that a bilateral agreement cannot provide. Until those outputs appear, this is evidence of expanded access—not evidence that Anthropic's systems are safe or that evaluator shopping has been solved.[1][2]
The first embedded report finds a faster research trend
METR's 5 November report changes the evidence base. Its embedded assessment ran from 12 August to 29 October and examined Anthropic's internal AI R&D capability index, model benchmark scores, analyses of researcher speed-up, forecasts and staff interviews. METR estimates that the trend in the company's capability index has accelerated by 1.9 times since October 2025. Its reported 95% confidence interval is 1.5 to 2.8 times, and alternative analytic choices produced estimates from 1.1 to 2.9 times. In plain terms, the evaluator believes Anthropic is now making relevant research progress at about twice its previous pace, but the plausible range remains wide.
METR says it would not be surprised if the current acceleration were between 1.3 and 3 times and judges that roughly 80% of the present acceleration may be due to AI. That 80% figure is an informed attribution, not a directly measured causal effect. The assessment found that the faster trend was not explained by faster growth in training compute, but other factors—hiring, organisational changes, research priorities and measurement choices—could still contribute. The index also represents a selected view of AI R&D capability; it is not a direct measure of scientific discoveries, product quality or safety outcomes.[3]
Self-reported speed-ups are much larger than the measured trend
The report records self-reported AI uplift reaching about twelve-fold in some contexts, far above the 1.9-fold trend estimate. METR treats those reports cautiously because the respondents and tasks are not a representative sample, people may count time differently, and a local task speed-up does not translate one-for-one into organisation-wide progress. Researchers described AI mainly as assistance for coding, analysis and iteration rather than as a fully autonomous system making major research contributions from start to finish.
That gap matters for workers and decision-makers. A tool can make an individual experiment or code change much faster while bottlenecks remain in choosing worthwhile questions, checking results, coordinating teams and deciding what is safe to deploy. Conversely, repeated assistance across many small tasks can compound into a meaningful organisational acceleration even if no single agent runs the programme. The strongest public conclusion is therefore about trend and workflow, not autonomous discovery: METR saw converging signs that AI is materially speeding Anthropic's research, with substantial uncertainty around the size and cause.[3]
Redactions limit what outsiders can verify
The access arrangement enabled METR to inspect evidence that an outside API evaluation could not reach. It also exposes the central governance weakness identified in the original article. Anthropic required substantial evidence to be redacted before publication, leaving readers unable to inspect some internal metrics, benchmark details and supporting material. METR publishes its judgement, intervals and methodological description, but the public cannot fully reproduce the analysis or test whether omitted evidence would change the result.
This does not make the report worthless: independent evaluators often need protected access to commercially sensitive systems, and publishing a bounded conclusion can be more informative than publishing nothing. But confidence should track auditability. A stronger next step would release de-identified series, prespecified analytic choices and enough benchmark detail for another qualified assessor to reproduce the acceleration estimate under secure conditions. Regulator access and a public log of redaction categories would also help distinguish legitimate confidentiality from evidence that is merely uncomfortable to disclose.[1][3]