Readable role signals appear before actionable prompt injections in AI agents
Readable Before Actionable: Causal Tracing of Indirect Prompt Injection
Artificial Intelligence
Summary
Large language models sometimes follow hidden commands found in outside data—a problem called indirect prompt injection. The authors show that the model’s understanding of its role or task can be detected before it actually changes its actions. They studied how changes inside the model affect behavior and found that readable signals in the model come earlier and are separate from the points where the model changes what it does. This means certain signals can be seen but don’t immediately make the AI follow unwanted instructions.
What this means in practice
- •For ai security teams: Identify early internal signals in language models that reveal attempts of indirect prompt injections to improve attack detection.
- •For ai system developers: Design models and tools that monitor readable role signals separately from actionable behaviors to better resist harmful injected commands.
Authors
Zhe Yu, Wenpeng Xing, Xingxing Yang, Meng Han
Abstract
Indirect prompt injection causes LLM agents to follow commands embedded in external data. A probe may distinguish instructions from data without identifying a state edit that changes the next action. We study this gap through counterfactual role probes, component-wise activation patching, and separate interventions on AgentDojo trajectories. Role decoding survives changes in content and format. In controlled Qwen tests, it precedes strong tool-choice effects from patches along an independently estimated role direction. On AgentDojo, directions estimated from hijacked and resisted training trajectories reduce attack success at pre-action and injected-span positions, but have little effect at random positions. In longer Qwen trajectories, single-position edits become less effective at later layers; span-wide and repeated edits reduce attack success on the same evaluation set. Removing the learned channel subspace preserves role decoding, yet effective intervention directions transfer poorly across the tested channels. These findings distinguish a readable role signal from an effective behavioral intervention: depth matters in controlled tool choice, while position and context also matter in attack trajectories.