AI agent networks measured for group agreement and decision accuracy

Measuring Consensus in AI Agent Networks

Multiagent Systems

Summary

When many AI agents work together and share information, it can be hard to tell if they really agree or reach the best decision. The authors created a way to watch how the agents’ opinions change and how much they share, helping to understand when they truly converge on an answer. They tested this on tasks like staffing decisions and fixing information conflicts, finding that sharing explanations helps accuracy but that sometimes agents still disagree even if the final answers are good. Their method helps separate true agreement from just reaching a correct answer based on what information is shared.

What this means in practice

  • For team managers: Assess how well AI systems coordinating decisions actually agree before relying on their outputs in group task settings.
  • For customer support teams: Improve understanding of when AI assistants collaboratively reached reliable answers versus when disagreements remain hidden.

Authors

Jin Gao

Abstract

As AI agents become increasingly connected through networks and take on tasks that require negotiation and joint decision-making, a systematic approach is needed to assess how their views converge and what information supports their decisions. We present an evaluation framework that treats consensus formation as the evolution of publicly stated positions toward a shared region, using accumulated communication history as context. The framework tracks position trajectories, group dispersion, and information expressed in public messages. Two case studies examine staffing decisions based on distributed facts and decisions requiring agents to reconcile duplicate information and a cancelled contract. Results from 1,872 completed runs associate sharing explanations with more accurate group decisions, while the second task reveals persistent disagreement despite accurate aggregate answers. Membership-change experiments provide a complementary evaluation. By separating convergence among agents from progress toward a task's full-information answer, the framework relates decision accuracy and bounded disagreement to the information available during the exchange.