Multi agent llm interactions reveal detailed information sharing patterns

Beyond Final Outcomes: Quantifying Claim-Level Information Dynamics in Multi-Agent LLM Interaction

Computation and Language

Summary

When multiple AI agents powered by large language models talk to each other, they share, change, and sometimes forget information. The authors created a method called RIVER that breaks down these conversations into small claims and measures how much the agents agree, explore new ideas, and remember past information. They tested different ways the agents interact in a news generation task and found that repeated pooling makes the agents agree more but limits variety, while passing messages along creates more diverse information but less agreement. Overall, the way information flows affects how well the group works together more than just looking at their final answers.

What this means in practice

  • For multi agent system developers: Design and evaluate multi-agent LLM interactions using claim-level metrics to balance agreement and idea diversity within collaborative AI teams.
  • For automated content generation teams: Optimize news or text generation workflows by selecting interaction strategies that preserve novelty or enhance consensus among AI agents as desired.

Authors

Haocheng Jiang, Herun Wan, Yongyuan Peng, Minnan Luo

Abstract

When LLM-powered agents interact, information is repeatedly shared, expanded, and forgotten, yet multi-agent evaluation typically focuses on final task outcomes. We study such interaction as claim-level information dynamics and propose RIVER, which decomposes agent outputs into atomic claims and evaluates three complementary dimensions: convergence, exploration, and retention. Convergence captures cross-agent claim correspondence and stance alignment; exploration captures semantic breadth and extension beyond the source; and retention tracks source preservation and the persistence of new claims across stages and lineages. We evaluate five interaction strategies on 40 controlled news-generation cases. Repeatedly pooled interaction promotes convergence but narrows the information space, whereas distributed relay yields broader, less shared information. Further comparisons show that method-specific constraints increase exploration while weakening convergence and retention. Our findings suggest that collective information dynamics are better explained by underlying information-flow mechanisms than by simple downstream performance.
View PDFOpen arXiv