Meta-Moderator: Empowering Multi-Agent Debate with Meta-Cognition
2026-08-24 • Computation and Language
Computation and Language
AI summaryⓘ
The authors study how group discussions among AI language models can be improved by better moderation. They propose Meta-Moderator, a trained system that watches the debate, decides when it's useful to keep going, and knows when to stop and pick the best answer. Unlike previous methods, Meta-Moderator learns from outcomes and isn't fixed or untrained. Tests show it manages discussions more efficiently and makes more reliable final decisions across different tasks.
multi-agent debatelarge language modelsmeta-cognitionpolicy optimizationdeliberation controlevidence aggregationoutcome-driven trainingAI moderationdecision layerstask transfer
Authors
Wentao Hu, Zhuoyue Wan, Jinhao Shen, Chen Jason Zhang, Xiaoyong Wei, Qing Li
Abstract
Multi-agent debate can improve large language model reasoning by eliciting diverse hypotheses and critiques, yet its performance is often constrained by weak moderation. Common pipelines rely on fixed budgets, agreement-based stopping, or untrained judges, leading to redundant deliberation and unreliable evidence aggregation. We cast moderation as a meta-cognitive process, monitoring debate utility, controlling deliberation, and adjudicating a final answer, and introduce Meta-Moderator, a learnable framework that dynamically regulates debate and decides when to finalize an answer. Meta-Moderator is trained independently of the debaters via outcome-driven policy optimization, making debate regulation an explicit capability rather than an incidental effect of prompting. Across five benchmarks, Meta-Moderator outperforms widely used decision layers and transfers across tasks and system configurations. Further analyses show that it allocates debate more selectively and reduces mis-aggregation after informative hypotheses appear.