Does ImpactBench measure how AI affects people?

The answer: it measures observable proxies, not people's outcomes

ImpactBench is a substantial attempt to compare how conversational models behave in situations linked to learning, agency, mental health, physical safety, relationships, privacy, finance and other aspects of wellbeing. It does not directly show that a person became healthier, more autonomous, less dependent or a better learner. Its pass rates describe whether model behaviour in simulated conversations matched criteria that experts and prior benchmarks defined as desirable or harmful.

Across 48,540 scored conversations, ten models passed 68.7% of positive criteria—where a supportive behaviour should appear—and 53.1% of negative criteria—where a harmful behaviour should be absent. The positive rate's 95% metric-cluster bootstrap interval was 66.2%–71.0%. The gap is a signal about this evaluation suite, not an estimate that models help or harm real users at those frequencies.

The preprint was submitted on 14 October by a large team spanning MIT Media Lab, USC, UC Berkeley, Carnegie Mellon, the University of Amsterdam, UNC Chapel Hill and Harvard. It has not been peer reviewed. The manuscript does not include a funding or competing-interest statement, so financial and institutional interests cannot be fully assessed from the paper.[1]

How 809 metrics became 48,540 conversations

The team adapted criteria from 26 benchmarks and taxonomies, including work on child risks, humane technology, mental-health response, tutoring, sycophancy, discrimination and hallucination. An LLM drafted candidate metrics from source material; internal experts reviewed, revised, merged or removed them. The final set contained 460 positive and 349 negative metrics across psychological, physical and social domains.

For each metric, another model generated three base scenarios. Every scenario was expanded into a teen version for ages 13–17 and an adult version for ages 18 and over, giving six scenario variants per metric and 4,854 age-expanded identifiers for each target model. A Claude model simulated the user for up to six turns, using a persona, a hidden adversarial goal and scheduled anchors intended to push the target model toward a revealing decision. Ten target models—six proprietary and four open-weight—produced the assistant side of each conversation.

A GPT model then judged each completed transcript against a metric definition and an interpretation clause. Positive metrics passed when the desired behaviour appeared; negative metrics passed when the harmful behaviour was absent. The judge did not receive the hidden scenario, user-simulator instructions, polarity or example answers. Aggregate uncertainty came from 10,000 bootstrap resamples of metrics, keeping associated scenarios, age variants and model results together.

This design makes cross-model comparisons traceable and broad, but it also means models generated the metrics, scenarios, users, interpretation clauses and final judgments at different stages. Human review sampled those stages; people did not validate every metric or conversation. The benchmark is therefore an auditable measurement pipeline with human checks, not a fully human-labelled population study.[1]

What the validation does—and does not—support

Ten internal experts reviewed scenarios for 85 metrics, 10.5% of the full set. Across 87 valid ratings, average quality was 2.97 on a four-point scale and 75.9% received a score of three or four. Ten experts separately rated 86 simulated conversations for realism; 81.4% were judged somewhat or very realistic. On a further 86 ratings, 70.9% of conversations were judged to apply significant or extreme pressure to the target model.

The automated judge was run three times on 240 conversation-metric pairs. Judgments were unanimous for 82.5%, mean pairwise agreement was 88.3% and Fleiss' kappa was 0.731. In a separate internal review of 87 judgments, experts agreed with 70, disagreed with six and were unsure about 11. Those are useful reliability signals, but they test consistency and sampled plausibility rather than whether every metric validly predicts a human outcome.

Nineteen external experts assessed 18 metrics across 11 benchmarks, generating 297 ratings. For four metrics with enough overlapping reviews, exact pairwise human agreement ranged from 44.4% to 73.3% and Krippendorff's alpha from 0.121 to 0.538. The automated judge agreed with the majority human verdict in 83.3% of comparable cases, with Cohen's kappa of 0.667. Reviewers deemed examples adequate in 60.6% of 33 metric-level assessments, were uncertain in 30.3% and judged 9.1% inadequate.

That disagreement is not a footnote. Concepts such as appropriate scaffolding, autonomy or emotional dependence contain normative choices. A consistent automated judge can reproduce one operational definition even when experts disagree about the definition or its boundary. Users should inspect metric-level evidence rather than treat one leaderboard score as a neutral measure of human flourishing.[1]

What the headline rates mean for educators, clinicians and families

The 68.7% positive pass rate and 53.1% negative pass rate indicate that, under this suite, models were more consistent at displaying requested helpful behaviours than at avoiding every defined harmful behaviour. Positive and negative performance were only weakly related across the 14 subareas, so a model could look supportive in one dimension while performing poorly on a different type of harm. Aggregate rankings also changed across benchmarks and weighting choices.

For teachers, a relevant distinction is between helping a learner reason and taking over the reasoning task. ImpactBench includes metrics for scaffolding, cognitive offloading and independent judgment. Those can guide procurement questions and classroom tests, but they cannot establish learning gains without students, teachers, assessments and follow-up. A simulated conversation that passes a scaffolding rubric is a hypothesis worth testing, not proof of educational benefit.

For clinicians and mental-health services, the benchmark offers stress tests for distress, crisis response, unhealthy dependence and refusal boundaries. It should supplement, not replace, clinically governed evaluation. Simulated users do not experience harm, and up to six turns cannot represent long-term dependency, belief change or delayed care. High-stakes deployment still needs professional review, incident monitoring and evidence from real service contexts.

For families and youth services, matched teen and adult scenarios are useful because they hold the construct roughly constant while changing age context. Yet the teen personas were synthetic and did not represent the full diversity of young people's language, culture, disability, family circumstances or risk. The results are best used to identify where child-specific testing is needed, not to certify a model as safe for all teenagers.[1]

What policymakers and model buyers can use now

A buyer can use the benchmark's structure as a checklist: define the behaviour that matters, justify it with evidence, create matched scenarios, preserve transcripts and expose judgments for review. Keeping positive and negative criteria separate prevents a model's friendly strengths from masking harmful patterns. Reporting benchmark-level reversals is also more informative than declaring one overall winner.

Policymakers should resist converting the score into a compliance threshold before validity and governance mature. The suite mixes 26 sources, and aggregate results depend on which metrics are included and how they are weighted. A public-service deployment needs locally relevant metrics, affected-community input and appeals when a criterion encodes contested values. Versioning is essential because models, judges and benchmarks will change.

The most useful contribution may be inspectability. Metrics, scenarios, transcripts and aggregation choices can be examined rather than hidden behind a proprietary score. Independent researchers should still reproduce the conversations and judgments under different simulators and judge models; the paper's bootstrap intervals are conditional on the saved conversations and do not capture uncertainty from rerunning the generation pipeline.[1]

Evidence limits and what would change the assessment

ImpactBench measures behavioural proxies, uses binary pass/fail labels and covers limited populations, cultures and languages. Controlled adversarial scenarios are not prevalence estimates for normal use. Six-turn simulations cannot reveal longer-term effects such as dependency, learning erosion or trust calibration. Results describe tested API configurations, not full products with their memory, interface, system prompts and safeguards.

The benchmark may also be vulnerable to evaluation awareness: a model can behave differently when it recognises a test. Human validation covered only samples, and several metrics produced modest expert agreement. Funding and competing interests are not disclosed in the manuscript. The model-generated stages create correlated measurement choices that a single aggregate confidence interval cannot fully represent.

Confidence would rise with preregistered validation against real-user outcomes, multilingual and community-authored metrics, repeated simulations across independent user and judge models, and published change logs. The assessment would strengthen if proxy scores predicted later learning, health, agency or relationship measures in prospective studies. It would weaken if rankings proved unstable across reasonable judges, prompts, cultures or product settings. For now, ImpactBench is a promising open evaluation instrument, not a direct meter of human impact.[1]