Virtual sensing improves analysis of multi-agent language systems
AgentSoV: Virtual Sensing for LLM Multi-Agent Systems via Stream of Variation
Multiagent Systems
Summary
Large language model (LLM) multi-agent systems are groups of AI agents that work together to solve complex problems, but it can be hard to understand why they sometimes fail. The authors introduce AgentSoV, a tool that treats these systems like dynamic processes to track how information and errors flow between agents. This approach lets users identify where problems start, how different parts influence outcomes, and where to best place sensors to monitor the system without rerunning it. Their method works across different system designs, helping improve the reliability and understanding of multi-agent AI.
What this means in practice
- •For ai system engineers: Detect and locate failures in complex multi-agent AI systems without rerunning them, improving debugging efficiency and reliability.
- •For software reliability teams: Optimize sensor placement to monitor AI workflows effectively under resource constraints by analyzing information flow between stages.
Authors
Congjing Zhang, Siyuan Xu, Yunfei Bai, Ke Ren, Xiaoning Qian, Shuai Huang
Abstract
Large language model (LLM) multi-agent systems (MASs) solve complex tasks through interacting agents, yet their reliability is difficult to analyze because failures depend on both individual agents and how information and errors propagate across the system. Existing approaches largely treat this as post-hoc failure attribution, but do not provide a common model across heterogeneous MAS architectures or support broader analysis of information influence and observability. We introduce AgentSoV, a virtual-sensing framework that models MAS execution as a stochastic dynamic system through Stream of Variation (SoV). AgentSoV maps heterogeneous trajectories into diagnostic stages, constructs virtual measurements from observable outputs, and learns cross-stage variation propagation with a state-space model. Its representation is architecture-aware in stage dependencies yet architecture-agnostic in analytical language, supporting chain-structured, orchestration-based, and directed acyclic graph (DAG)-structured MASs. Without re-executing the MAS, AgentSoV supports three system-level analyses: failure attribution identifies where abnormal variation enters; stage-input influence quantifies how strongly intermediate inputs affect downstream outcomes; and virtual-sensor placement selects complementary observations under a monitoring budget. Experiments across multiple MAS architectures demonstrate a unified approach to fault localization, information influence, and observability. AgentSoV reframes agentic execution as a dynamic system to model rather than merely text to inspect, providing insights for MAS structure and component design.