Agenttriage provides consistent security diagnosis across different agent platforms
AgentTriage: Replayable Security Diagnosis Across Heterogeneous Agent Runtimes
Artificial IntelligenceCryptography and Security
Summary
When computer programs called tool-use agents perform actions like writing files or running code, it’s hard to understand exactly what caused security problems because different systems record events in different ways. The authors created AgentTriage, a tool that standardizes and records these events so security issues can be replayed and diagnosed consistently across multiple platforms. This means security claims can be checked again later without needing human judgment, improving reliability and clarity. They tested AgentTriage on five different systems and showed it performs better than existing methods.
What this means in practice
- •For security operations teams: Diagnose and verify security events from automated agents across multiple runtime environments with consistent evidence tracking.
- •For devops engineers: Integrate cross-platform security event tracing into deployment pipelines to improve incident response accuracy for automated scripts and tools.
Authors
Jie Ying, Xutao Zhang, Ningyuan Chen, Mengce Zheng, Jun Li, Zhongjie Zhu, Tiantian Zhu
Abstract
When tool-use agents write files, execute code, or send data, their final responses do not explain the resulting security events, and heterogeneous runtimes expose the relevant evidence through incompatible interfaces. We present AgentTriage, an executable contract for replayable security diagnosis across heterogeneous runtimes. The contract admits conclusions whose claims cite canonical runtime events and requires abstention when mandatory evidence is missing. Runtime overlays normalize native observations, while a deterministic pipeline joins invocation and result lifecycles, constructs a typed provenance graph, selects a policy-decisive slice, and binds claims to recorded evidence. Given the same normalized trace, overlay version, and pre-run policy oracle, the diagnosis can be recomputed and its claims re-checked without language-model judgment. Across five runtimes, one contract captures all 159 applicable executions, with an audit of native and synthesized identifiers. On 120 adjudicated cases, AgentTriage achieves 0.858 grounded joint diagnosis, 0.966 required-evidence coverage, and a 0.036 unsupported-claim rate, improving joint diagnosis by 0.175 (95% CI [0.092, 0.258]) over the strongest same-schema baseline. These results quantify the precision of replayable diagnosis and the observation boundaries that require abstention. We release an open-source implementation and selected sanitized evidence at https://github.com/AsnGs/agenttriage-open-source.