AI debate protocol ensures sensitive and correct judgment every time
How to Have a Sensitive Debate: An Instance-Optimal Protocol for AI Debate
Artificial IntelligenceComputational ComplexityComputer Science and Game TheoryMachine Learning
Summary
It can be hard to check if powerful AI systems give the right answers to tough questions. The authors improve a method called AI debate where two AIs argue over a question to find the truth. Their new rules guarantee that the debate outcome is always correct, even in the hardest cases, and that both AIs do their best by being honest. They also prove that their method is the best possible when judging claims by asking humans only simple questions.
What this means in practice
- •For ai safety engineers: Design debate protocols that guarantee correct oversight even in worst-case AI outputs with minimal human supervision.
- •For automated verification teams: Develop verification tools using debate approaches that ensure truthful answers in complex problem-solving scenarios.
A theory result. No direct application yet.
Authors
Jiawei Li, Zhiyang Xun, Lijie Chen, Jonah Brown-Cohen
Abstract
As powerful AI systems reach and sometimes surpass the abilities of human experts across a range of cognitively demanding tasks, the problem of accurate oversight and supervision of these systems has become increasingly urgent. One promising approach is AI debate, which seeks to leverage a debate between two powerful AIs to break complex questions down into simpler claims that can be easily judged directly. Theoretical work on debate has formalized this intuition in the language of computational complexity theory, where the goal is to design protocols (i.e., rules of the debate game) that provide rigorous guarantees on correctness for judging solutions to complex problems with limited supervision. Specifically, the current best protocol has been shown to work for all problems that have sufficiently stable decompositions into subproblems. In this paper, we design a new protocol for this same class of problems that improves on the prior work in several ways. First, correctness holds in a worst-case rather than an average-case sense. Second, being honest and correct is a dominant-strategy equilibrium for both debaters, rather than a Stackelberg equilibrium. Finally, we prove black-box lower bounds, showing that our new protocol is instance-wise optimal. That is, no protocol for this class of problems can outperform ours while making only black-box queries to human judgments. We obtain these results by relating the notion of stable problem decompositions to the concept of fractional block sensitivity from query complexity.