Keeping audit coin secret prevents manipulation of AI action checks

Blindfold: Keeping the Audit Coin Secret from LLM Agents

Cryptography and SecurityArtificial IntelligenceMachine Learning

Summary

This paper looks at how to fairly check whether an AI agent is making mistakes by auditing some of its actions randomly. The researchers found that for these checks to be reliable, the AI agent must not be able to see or influence which actions are being audited before it acts. They call this rule pre-commit noninterference. If the AI can guess or detect the audit, it can cheat, making the audit unreliable. Their experiments show how even small leaks, like timing delays during audits, let the AI learn which actions are audited, defeating the purpose. Using a fresh secret for each audited action can fix this problem.

What this means in practice

Authors

Wei Zhu, Meiyun Zuo, Huiping Sun

Abstract

Audit-based certification bounds the violation rate of an LLM agent by auditing a random fraction $\varepsilon$ of actions and reweighting the audited labels; its validity is meant to rest only on the defender's audit coin. We show that it also rests on a confidentiality property: the coin must not interfere with the agent's view before the action commits, at a time the agent chooses. We call this pre-commit noninterference (PNI) and measure its violation by $D_{\mathrm{A}}$, the total-variation distance between the agent's views under the two coin values. PNI is necessary and sufficient for an unbiased certificate against adaptive agents, the worst-case bias is $(1-\varepsilon)D_{\mathrm{A}}$, and no procedure that sees only audited labels can pay less. Leakage composes across steps, a resource shared across episodes opens a channel of computable size, and raising the audit rate cannot restore validity. On a monitor we built, a silent audit changes no output but adds latency that tool timestamps expose, and the agent can enlarge it. A real LLM agent follows an injection that names a visible audit notice (generic, tool, and timing clauses have no effect); over that channel the plain certificate fails on measured labels and, by a post-hoc analysis, in a live run on a pre-registered schedule, while a silent audit restores it. A fresh coin per audited call closes the timing channel; padding only narrows it.