AI Debate Aids Assessment of Controversial Claims
Salman Rahman, Sheriff Issaka, Ashima Suvarna, Genglin Liu, James Shiffer, Jaeyoung Lee, … (+8 more) — 2025-06-02 — University of Washington, Microsoft Research, UCLA, NYU — arXiv
Summary
Empirically tests whether AI debate improves human judgment on controversial factual claims about COVID-19 and climate change, comparing debate protocols (two AIs arguing opposing sides) versus consultancy (single AI advisor) with both human and AI judges.
Key Result
Debate consistently improves human judgment accuracy by 4-10% over consultancy across controversial claims, with AI judges using human-like personas achieving 78.5% accuracy compared to 70.1% for human judges.
Source
- Link: https://arxiv.org/abs/2506.02175
- Listed in the Shallow Review of Technical AI Safety 2025 under 2 agenda(s):
- black-box-make-ai-solve-it — Black-box safety (understand and control current model behaviour) / Iterative alignment
- debate — Make AI solve it