CounterSteer defends language models from hidden prompt instructions
CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering
Cryptography and SecurityArtificial Intelligence
Summary
Some language AI models can be tricked into following secret instructions hidden inside text they get from outside sources, which can lead them to behave in unwanted ways. This paper introduces CounterSteer, a method that changes the way the model processes all output from external tools to block these hidden commands without needing extra training or detection steps. The authors tested CounterSteer on several open-source models and found it greatly reduced attacks while keeping most normal uses unaffected. However, some types of attacks are only partially stopped, suggesting that additional controls may be needed for full protection.
What this means in practice
- •For ai platform engineers: Implement runtime defenses that suppress hidden instructions in retrieved text during model inference to improve safety without retraining models.
- •For automated customer support teams: Protect AI assistants from being hijacked by malicious inputs embedded in external data sources while maintaining normal support quality.
Authors
Mark Russinovich
Abstract
Indirect prompt injection makes an LLM agent treat untrusted retrieved text as instructions. We present CounterSteer, an inference-time defense that suppresses this behavior inside the model. Per model, a five-step recipe fits a residual-stream direction from paired episodes differing only in whether an embedded instruction is followed, and retains it only if it passes pre-specified causal and capability gates. At deployment, the direction is subtracted from every tool-result token during prefill. The edit is always on--there is no detection decision to evade--and requires no fine-tuning, auxiliary model, or added tokens, only white-box serving and tool-result span boundaries. Across five open-weights models (8B-106B, five vendor lineages), held-out attack success falls from 0.21-1.00 undefended to 0.00-0.17 defended, and AgentDojo compromise rate from 0.10-0.49 to 0.006-0.079, at 93-100% typography-normalized benign utility, with larger task-dependent costs when reasoning over steered content. A benchmark-level adaptive attacker reaching 0.67-0.73 undefended is held to roughly a quarter of that on the two most deeply evaluated models. Among the defenses we measured on capable models, those achieving lower compromise rates either lost 22-89% of benign utility or fine-tuned the served weights. White-box gradient attacks through the deployed vector compromise at most 2 of 52 episodes, and none of 2,052 replayed human red-team attacks succeeds. CounterSteer largely neutralizes instructional takeover: a black-box framing search cracks 3 of 18 development samples. Parameter manipulation--attacker-chosen arguments in otherwise legitimate calls--is only partially resisted (13 of 18); the decision becomes linearly readable at argument emission but not at the examined pre-generation sites, and is not removed by the tested prefill- or decode-time steering, motivating argument-provenance controls.