Visual rendering reduces prompt injection attacks on language models
Render Before Reading: Visual Rendering as a Prompt Injection Defense
Cryptography and SecurityArtificial Intelligence
Summary
Large language models can be tricked by harmful instructions hidden in text, causing them to behave unexpectedly. The authors found that when these instructions come as images rather than text, the models are less likely to follow them. This is because the models were mainly trained to obey text instructions, not visual content. By turning risky text into images before showing them to the model, the authors created a way to block harmful instructions without retraining. Their testing showed this method works well against strong attacks while keeping normal features useful.
What this means in practice
- •For ai security teams: Protect multimodal language models by converting untrusted text prompts into images to reduce prompt injection risks without retraining models.
- •For chatbot developers: Improve chatbot safety by integrating visual rendering steps to filter harmful textual commands before interpretation by language models.
Authors
Jie Zhang, Andrei Baroian, Jan N. van Rijn, Avital Shafran, Florian Tramèr
Abstract
Large language models are vulnerable to prompt injection attacks, where third-party adversarial content can hijack the model's behavior. In this paper, we study the role played by the adversarial data's input modality, and identify a systematic asymmetry: multimodal LLMs are more likely to follow adversarial instruction when they appear as text than when the same instruction is delivered through a non-textual channel (e.g., as an image). We hypothesize that this modality gap arises from text-centric instruction tuning, which teaches models to obey textual instructions while treating other modalities mainly as content to parse or describe. We then demonstrate how this gap can be turned into a training-free defense, by rendering all untrusted payloads as typographic images (or audio) before they reach the model. Across ten models and two prompt injection benchmarks (DirectInject and AgentDojo) we show that our defense Pictionary consistently reduces attack success rates even against the strongest adaptive attacks and human red teamers, while largely preserving benign utility. We further show that benign fine-tuning on image-rendered instructions erodes the modality gap, tracing it to the text-centric instruction-tuning distribution.