Visual memory attacks can influence AI even after removal
Visual Memory Attacks Can Persist Through The KV Cache
Cryptography and SecurityArtificial IntelligenceComputer Vision and Pattern Recognition
Summary
Modern AI language models often use images and text together to understand long conversations. This paper shows that harmful images can secretly trick these models to behave in certain ways even after the images are no longer visible. The researchers developed special attacks that hide in the AI’s internal memory, making the bad influence last longer than expected. They tested their ideas on a real language-and-vision AI and found the attacks stayed effective over many conversation turns.
What this means in practice
- •For ai security teams: Detect and mitigate persistent visual attacks that remain in model caches even after removing their source images.
- •For ai developers: Test robustness of language-vision models against adversarial images that persist in the model’s internal memory over long conversations.
Authors
David Dobre, Leo Schwinn, Gauthier Gidel, Spandana Gella, Perouz Taslakian, Pierre-André Noël
Abstract
Modern language model systems operate autonomously over increasingly long contexts containing untrusted text and images. Can an adversarial input continue to steer a model even after that input is removed from its context? We show that attacks can be trained to persist through the key/value (KV) cache of subsequent tokens, allowing adversarial influence to outlive direct access to its source.We consider the Visual Memory Injection (VMI; Schlarmann and Hein, 2026) attack setting, in which an adversarial image that stays in the context plants a hidden backdoor: the model behaves normally until a chosen trigger elicits an attacker-chosen response. We first demonstrate persistence in this setting with optimized soft prompts, which remain effective after we mask the prompt from attention. We then introduce Persistent Visual Memory Injection (P-VMI), which optimizes images to preserve this adversarial behaviour after they are masked from attention. These attacks persist over conversations substantially longer than those used during optimization. On Qwen3-VL-8B-Instruct, P-VMI achieves up to approximately $90\%$ target success in its strongest configuration and remains effective under a stricter removal setting that exposes the image only on the first turn. A cache-swap ablation localizes the persistent influence to the KV cache. Finally, we show that these attacks can be trained to survive compaction that retains the KV cache of a summary generated by the same model, demonstrating that adversarial behaviour can persist in cached state without continued access to its source.