Multi-agent AI messages can wrongly change correct answers
When Upstream Messages Override Correct Answers: A Controlled Study of Multi-Agent LLM Collaboration
Artificial Intelligence
Summary
Sometimes, different AI agents work together by sending messages to solve hard problems. The study shows that helpful messages can fix mistakes, but wrong messages from one agent often cause another agent to ditch its correct answer. In most cases where harm happens, the second agent just copies the incorrect answer it got. The researchers suggest only trusting messages when the sender is reliable and the receiver lacks strong evidence.
What this means in practice
- •For ai system designers: Improve multi-agent AI communication by filtering unreliable messages to reduce error propagation in collaborative tasks.
- •For software engineers: Build multi-agent systems that selectively accept messages based on evidence and sender reliability to maintain answer accuracy.
Authors
Yaxin Gong, Gangyi Zhang, Chongming Gao, Leyang Shen, Chenxiao Fan, Jiakai Wang, Dong Wang, Yang Liu, Wenjie Wang, Xiangnan He
Abstract
Multi-agent LLM systems rely on message passing among specialized agents to accomplish complex tasks. However, an upstream agent may provide useful information or an incorrect answer that causes a downstream agent to override a correct answer supported by its own evidence. Prior work has not clearly separated the benefits of communication from the damage caused by incorrect messages. We study this problem with controlled experiments across five benchmarks and five receivers, keeping the downstream task and evidence fixed while comparing answers under three conditions: no message, the upstream agent's original message, or a message with the opposite conclusion. Our experiments reveal three key findings. First, messages often help when the downstream agent would otherwise answer incorrectly. Second, messages can also hurt: when the downstream agent would answer correctly without a message, an incorrect upstream message changes the answer in up to 32% of cases. Third, in 94% of audited harmful cases, the downstream agent copies the upstream's specific wrong answer--a pattern we term answer substitution. Removing unreliable messages recovers part of the lost accuracy, suggesting that communication should be selective based on upstream reliability and the evidence already available to the downstream agent.