Adding ai advisers increases visible disagreement and impacts trust
A latent dimension of Condorcet's jury theorem for multiple AI advisers
Computers and SocietyArtificial IntelligenceHuman-Computer InteractionMultiagent Systems
Summary
When asking many AI advisors the same question, having more advisors usually makes the group decision more accurate. But the authors show that as you add more advisors, it's almost certain some will disagree visibly, even if the group is correct overall. This disagreement happens faster than the group becoming more reliable if the advisors are only moderately accurate. So, seeing conflicting answers does not always mean the AI group is failing, but it does affect how users should understand the advice.
What this means in practice
- •For ai system designers: Design AI assistant panels by deciding how many advisers to query and how to show their disagreements meaningfully to users.
- •For customer support teams: Structure multi-adviser systems to better interpret and present conflicting AI recommendations for clearer user communication.
A theory result. No direct application yet.
Authors
Kazutoshi Sasahara, Aoi Naito, Ryo Fujie
Abstract
When the same question is asked of multiple AI advisers, as in self-consistency and LLM-as-a-judge panels, Condorcet's jury theorem predicts that adding independent, competent advisers makes the majority more reliable. The theorem, however, has a latent dimension when viewed from the user's vantage: adding advisers also makes disagreement more visible. A binomial model reveals that this ``visible dissent'' becomes nearly inevitable as the number of advisers grows, and that reliability and disagreement both approach certainty but at different convergence rates. The two rates cross at an adviser accuracy of 4/5 (0.8). Below this value, visible dissent approaches certainty faster than reliability and, with enough advisers, becomes more likely than a correct majority. Even ideal panels of independent and competent advisers can be correct in aggregate but appear divided; such disagreement does not by itself indicate aggregation failure. The way advisers split also provides a common basis for predictive multiplicity, reconciliation load, and reliance miscalibration. These results indicate two distinct decisions when using multiple AI advisers: how many advisers to consult and how their verdicts should be presented and interpreted.