Multi-agent debate improves reasoning when proposals and checks work together
When Debate Helps: Proposal Supply and Verification-Aware Readout in Multi-Agent Reasoning
Artificial IntelligenceComputation and LanguageMachine LearningMultiagent SystemsSocial and Information Networks
Summary
Sometimes groups of AI agents vote on the best answer, but simple voting doesn’t always pick the right one. The authors explain that for debate among agents to work well, two things are needed: someone must suggest the right answer, and the system must be able to recognize it even if most votes say otherwise. They created a way to measure when the right answer is suggested but ignored, and designed a method to help identify correct answers more reliably. They also built agent communities that better cover possible answers, resulting in more accurate final decisions.
What this means in practice
- •For ai system developers: Create multi-agent reasoning systems that improve over majority voting by ensuring diverse candidate answers and better evidence evaluation.
- •For automated decision teams: Deploy debate frameworks that identify correct answers even when simple voting fails, enhancing trust in automated conclusions.
Authors
Zihao Zhao, Tunyu Zhang, Haizhou Shi, Yusong Zhao, Xinxi Zhang, Hao Wang
Abstract
Multi-agent debate can improve reasoning, yet often fails to beat simple majority voting. We argue that successful debate requires two distinct mechanisms: proposal supply must surface a correct answer, and readout must identify that answer when voting misses it. We formalize the first requirement through recoverable headroom, which measures cases where a correct proposal is available but the majority answer is wrong. For the second, we develop Latent Verification Debate (LVD), an accounting model in which candidate proposals receive answer-specific verification evidence before final generation. Controlled fixed-proposal interventions estimate this latent effect in equivalent peer-support units and show that correct evidence changes answer probabilities and generated decisions while proposal supply remains fixed. To improve proposal supply, we construct societies from neural-thicket agents using labeled and label-free coverage objectives. Across two backbones and matched-budget reasoning benchmarks, coverage-selected societies increase complementary proposal supply and improve aggregate accuracy in repeated stochastic evaluations. Round-level controls further show that interaction provides gains beyond applying the same finalizer directly to the initial proposals. These results identify proposal coverage and truth-sensitive evidence use as complementary conditions for debate to outperform voting. Code is available at https://github.com/Wang-ML-Lab/when-debate-helps.