Debate helped people correct mistaken beliefs—but did weaker AI judges improve?
Across five experiments, human judges improved after structured debate. Weaker AI judges often did not, while model diversity and the interaction format changed the result.
The short version
- What happened
- About 690 UK participants were recruited through Prolific and 507 remained after exclusions across five experiments; debate raised one human-judge result from 41.3% to 57.61% accuracy.
- Who it affects
- Organizations using AI panels for review could gain false reassurance if several similar models repeat the same mistake or persuade one another without checking external evidence.
- What remains unproven
- The unreviewed experiments used induced beliefs, binary factual claims, small artificial panels and facts that may be present in model training data; they did not measure real-world investigation quality or harm.
Keep in mind
The unreviewed experiments used induced beliefs, binary factual claims, small artificial panels and facts that may be present in model training data; they did not measure real-world investigation quality or harm.
Source analysis · 1 linked source. Primary analysed source date: 20 October 2026.
Read the full report · 3 min The evidence, methods, implications and limitations
Structured argument helped human judges revisit an induced belief
Researchers ran five experiments on belief correction using binary factual claims. Around 690 UK adults were recruited through Prolific and 507 remained after exclusions. Participants first saw evidence designed to induce a mistaken answer, then judged arguments presented as consultancy or debate. In one human-judge experiment, accuracy rose from 41.3% before debate to 57.61% after it; consultancy changed 51.25% to 48.75% and did not help.
Pooling multiple independent human judges also helped: majority voting rose from 43.06% to 62.50%, while confidence-weighted voting rose from 45.83% to 65.28%. Those results suggest that competing arguments and independent aggregation can expose an error under controlled conditions, but they do not show that every debate format produces truth.[1]
Model strength and panel diversity changed the AI result
The study tested five proprietary and three open models. In an unconditioned comparison, debate significantly improved GPT-5.2 from 84.85% to 90.4%. Claude Sonnet 4 fell from 84.38% to 76.56% after consultancy and from 85.26% to 78.95% after debate, with only the consultancy decline meeting the authors' significance threshold. Smaller open models did not consistently gain from argument.
Consensus also mattered. Deliberation reduced accuracy for homogeneous three-judge panels from 48.96% to 44.04%, while diverse seven-model panels improved from 58.26% to 64.29%. For safety teams, 'add debate' is therefore not a complete control: judge capability, independence, diversity and the aggregation rule all need evaluation. Realistic domains, unseen claims and external fact checking would change the assessment.[1]
The distinction that matters
Three ways the final judgment was produced
The study separates persuasive interaction from independent aggregation.
Consultancy
One advocate argues for one answer
Judge may inherit a one-sided case
Debate
Opposing advocates challenge each other
Contradictions can become visible
Panel
Independent judgments are combined
Diversity can matter more than discussion