LLM agent actions audited by query-based attribution models
Auditing Agent Actions through Query-Conditioned Attribution
Artificial IntelligenceComputation and Language
Summary
It can be hard to figure out why AI agents take certain actions, especially when those actions involve many steps and complex reasoning. The authors introduce a new way to ask specific questions about an agent’s decisions and get clear explanations of which parts of its history contributed to those choices. They build a large benchmark of questions and show a system that uses small, efficient models to find the most relevant evidence for an action quickly and accurately. This helps better track and understand AI behavior, even when the AI’s full workings are hidden behind an API.
What this means in practice
- •For ai system operators: Quickly identify why an AI agent took a specific action by tracing the relevant decision history based on targeted audit questions.
- •For security analysts: Track down the cause of unsafe or failing behavior in AI agents by linking observed actions to their prior inputs and decisions.
Authors
Yifan Liu, Praveen Venkateswaran, Abdulhamid Adebayo, Dong Wang
Abstract
LLM agents increasingly take consequential actions through interactions with users, policies, and external tools. Auditing these agents requires automated attribution of realized actions to their historical basis. However, existing attribution formulations do not provide question-specific traces for diverse auditing objectives. Additionally, when access to the acting model is limited (e.g., in API-only deployments), applicable methods commonly rely on costly input perturbations or external LLM analysis of complete trajectories. We therefore formulate $\textit{query-conditioned agent action attribution}, a new task that takes a natural-language auditing query as input and recovers the source and ordered intermediate evidence for the query-specified aspect of an action. We instantiate this task with $A^3Bench$, a benchmark comprising 1,396 auditing queries across policy basis, parameter provenance, failure propagation, and unsafe-behavior tracing. To enable efficient, query-specific attribution, we use small open-weight models as attribution proposers that combine query-conditioned gradient saliency with query-semantic relevance to rank history units. Our proposer consistently achieves stronger source and evidence rankings at lower inference cost than open-weight baselines, improving source MRR by up to 40.9\% and evidence MAP by 42.1\% with only two forward passes and one backward pass. Controlled evaluations confirm that our proposer improves attribution specificity by adapting its rankings to fine-grained changes in the auditing query. Building on a proposer ensemble, our end-to-end system surpasses the strongest frontier-model baseline in source accuracy (64.5\% vs.\ 60.4\%) while reducing empirical deployment latency by 29.9\% relative to the fastest frontier API baseline. Code and data will be released after the initial review period following final validation and cleanup.