Corrupted graphs cause errors in language agent action verification
Who Verifies the Graph? Misspecification Attacks on Causal Action Verification for Language Agents
Artificial IntelligenceMachine Learning
Summary
Language agents use graphs to check if their planned actions make sense and are safe. This paper shows that if these graphs have small mistakes or are tampered with, the agent can execute wrong or harmful actions without noticing. The authors tested a defense that spots these attacks by rechecking a random sample of actions, which catches errors but also rejects many good actions. Fixing these issues requires extra tests, making the system slower and less useful. Ensuring safety against wrong actions is easier than recovering lost good actions.
What this means in practice
- •For language ai developers: Detect and defend against hidden errors in action verification caused by wrong causal graphs in language agents.
- •For software quality assurance teams: Audit and validate the correctness of decision-making tools that rely on causal graphs to reduce harmful automated actions.
Authors
Fabio Rovai
Abstract
Causal action verifiers gate an agent's state-changing tool calls by checking whether each proposed intervention is identifiable against a committed action-state graph, and they issue a certificate that carries the identification argument and a one-sided lower confidence bound. One such verifier, CIVeX, reports zero false executions on a confounded tool-use benchmark. We red-team it by corrupting only the committed graph. Omitting a single bidirected edge takes it from zero false executions to 15.3% at the benchmark's published confounding strength, with 91% of its executions harmful and utility falling from +2.27 to +0.35. Reversing one arrowhead, so that a mediator is committed as a confounder, gives 48.9% false executions and no correct ones. Every one of these actions carries an internally valid certificate. An attestation step that tests each observationally certified execution against a bounded randomised sample detected both attacks, with 2 false alarms in 555 executions on a truthful graph; refusing what fails the test, or cannot be tested, gave zero false executions in every setting we measured. It does not restore beneficial execution: at the published strength 97.1% of beneficial actions are still never executed, because the same misspecification rejects them before attestation runs. Those rejections carry certificates too, and auditing them works, but its cost scales with the number of rejections rather than the number of executions. Recovering safety costs 127 experiments per 1,050 actions; recovering the lost value costs 614 more, at which point the audited verifier makes the honest graph's decisions on every instance and spends exactly its experiment budget. An audit that inspects only executions protects against wrongful action. Wrongful inaction has to be paid for separately.