Execution consistency improves judging multi-agent task compliance
Beyond Corrected Memory: Execution Consistency in Multi-Agent Systems
Artificial Intelligence
Summary
When multiple agents work together, shared memory logs their actions, but these correct records don’t always show if the agents did their jobs right. The paper defines new ways to check if agents really followed their tasks by looking beyond memory to how actions depend on each other and agree in the end. The authors use a tool called CAVERT that reads logs to spot when agents acted correctly or not, doing better than existing methods. This helps make sure agent actions in multi-agent systems can be judged reliably.
What this means in practice
- •For distributed system developers: Diagnose and recover from inconsistencies in multi-agent task executions by analyzing execution relationships extracted from system logs.
- •For automation engineers: Improve monitoring tools to verify if automated agents meet task requirements using richer evidence of execution consistency beyond simple memory records.
Authors
Zhe Yu, Zixuan Wang, Peidong Wang, Hehai Lin, Ruochen Zhao, Chengwei Qin
Abstract
Shared memory coordinates agents' actions, but correct records do not establish that those actions satisfy task requirements. Memory governance and failure diagnosis regulate or inspect recorded information; they do not by themselves establish whether it is sufficient to judge task duties. We define execution consistency through duties governing state use, information handoffs, and final-state agreement, with explicit evidence conditions for judging fulfillment. Our core claim is that identical retained records can correspond to compliant and violating executions under the same task rule. Controlled removal of evidence such as receipt, action dependence, or response validity leaves 82.4% of opposite-label pairs indistinguishable; restoration separates 97.9% of the merged pairs. Natural-log annotations identify the defined violations in actual executions. However, existing logs do not always explicitly represent the execution relationships needed for these judgments. To assess the definition's practical value, we use CAVERT, a framework for consistency diagnosis and recovery, to extract supported relationships from logs and apply these criteria. It consistently outperforms contract-prompted LLM and rule-based baselines in diagnosis across all 12 benchmark-executor settings. Under the same gate and executor limits, it also outperforms rule-guided recovery in all four evaluated environments. These findings identify execution evidence that agent-memory and execution interfaces should preserve for reliable judgment.