Certifying safe structured actions across multiple corrupted sources
Certified Multi-Source Integrity for Structured Agent Actions
Cryptography and SecurityArtificial IntelligenceMachine Learning
Summary
Sometimes AI agents perform important actions like paying bills, where parts of the action come from different documents or tools that might be tampered with by attackers. Existing protections don’t always check whether these combined pieces can be safely trusted together. The authors develop a way to only allow actions if all parts have strong, independent support that is hard to forge. They test their method on real-world examples like sanctions lists and software provenance, showing their system blocks attacks that fool current methods while approving correct actions when proper evidence exists.
What this means in practice
- •For enterprise security teams: Prevent unauthorized automated actions by certifying the combined trustworthiness of all evidence sources feeding into agent decisions.
- •For software supply chain managers: Improve confidence in software component origins by enforcing multi-source evidence checks that resist tampering and reuse of corrupted data.
Authors
Anmol Pandey, Aditya Jain, Liang Chen, Carsten Maple, Christo Panchev
Abstract
LLM agents increasingly take privileged, often irreversible structured actions, such as paying an invoice. They assemble each action from action-critical fields in documents and tool outputs that an adversary can corrupt, and indirect prompt injection can drive the model itself to extract attacker-chosen values. Current defenses gate on a source's trust label or certify free-text answer quality. None certifies the integrity of a coupled, policy-bound structured action under a corruption budget that accounts for shared upstream sources. We characterize when such an action is safely certifiable and give the maximally live safe certifier. It admits an action only when each field clears the rule its evidence structure supports: a bounded corruption radius over corruption-distinct evidence classes, counted by a minimum hitting set so that re-publishing or laundered copies cannot manufacture a quorum, deterministic reconciliation for complementary fields, and a trusted anchor where the evidence leaves a field single-sourced. We formalize two robustness notions, validate each mechanism by ablation, and measure how often the multi-source precondition holds on sanctions designations (70,966 entities) and software supply-chain provenance (450 packages). Under upper-bound proxies, genuine corroboration is a minority phenomenon in both, and naive attestation counting overstates it, since witnesses that look independent collapse to two corruption-distinct domains once shared origin is counted. Across five current models in a real agent loop, a realistic injection fools every model but one and a naive agent then executes the fraudulent action on most attacks. The certifier admits no unsafe action and recovers the correct value where corroboration permits, while action-gating and provenance baselines are broken in every world of our harness by some attack in its space.