Latent communication links increase safety risks among multi-agent systems
Safety of Latent Communication in Multi-Agent Systems
Artificial IntelligenceMachine LearningMultiagent Systems
Summary
This paper looks at how systems of agents communicate by sharing hidden signals instead of words, which makes communication faster and lighter. The researchers found that training these hidden links can unintentionally increase harmful behavior, even if the individual agents are designed to be safe. They also show how attackers can make these hidden links promote harmful actions without obvious signs. However, they present ways to fix compromised links and reduce risks without changing the agents themselves. The study highlights that keeping multi-agent communications safe means looking at the system as a whole, not just individual parts.
What this means in practice
- •For multi-agent system developers: Improve safety measures in multi-agent communication by monitoring and repairing latent communication links to prevent harmful compliance.
- •For autonomous vehicle teams: Detect and mitigate risks of hidden communication channels among autonomous cars to maintain safe collaborative driving without changing individual vehicle policies.
Authors
Muhammad Huzaifa, Sina Mavali, Thorsten Eisenhofer
Abstract
Latent communication enables multi-agent systems to exchange information directly in internal representation space, reducing the token, computation, and latency overhead of text-based communication. To this end, lightweight trainable links are introduced to map the sender's representations into the receiver's input space. In this work, we show that even benign link training can increase harmful compliance relative to text-based communication while the underlying safety-aligned agents remain unchanged. An attacker can amplify this effect by optimizing the links on harmful query--response pairs or poisoning otherwise benign training data. We further develop a reinforcement-learning attack that rewards harmful compliance alongside benign task performance without requiring harmful target responses. Across three communication topologies and four safety benchmarks, this attack raises the mean harmful-compliance score from 27.9 with benignly trained links to 76.9. Compared with direct supervised optimization, it also achieves higher average accuracy on two benign utility benchmarks. Adapting the rewards toward safer behavior also enables repair of compromised links, substantially reducing harmful compliance across all evaluated attacks without updating the agents. Overall, our results show that safety alignment requires considering the multi-agent system as a whole.