Memory systems in AI agents vary in vulnerability to poisoning attacks
MemoryDojo: Benchmarking Poisoning Through LLM Agent Memory
Cryptography and SecurityArtificial IntelligenceComputation and Language
Summary
Large language model (LLM) agents use memory to handle long tasks across multiple sessions, but this memory can be attacked by harmful instructions that get stored and used later, causing problems. The authors created MemoryDojo, a tool to test how different memory systems handle these attacks by tracking bad information over time and its influence on the agent's decisions. They found that all memory types can be compromised, but some are more resilient than others. They also developed a way to make attacks more effective to better understand the weaknesses in various memory designs.
What this means in practice
- •For ai product developers: Test how different memory architectures in LLM agents handle malicious input to improve system safety.
- •For security teams in ai companies: Evaluate and strengthen defenses against memory poisoning attacks in AI agents used for long-term interaction.
Authors
David H. Yang, Blake Bullwinkel, Amanda Minnich, Mohammad Mohammadi Amiri, Giorgio Severi
Abstract
LLM agents benefit from using memory systems to improve performance on long-horizon, multi-session tasks. However, memory introduces a critical attack surface. Malicious instructions encountered in one session may be stored and propagated through memory before being retrieved and acted upon in later sessions. Because memory systems differ in how they process and retrieve information, the downstream effects of poisoning can vary significantly across systems. To quantify the effects of poisoning across different memory architectures, we introduce MemoryDojo, a benchmark for tracing the lifecycle of poisoned information through agent memory and measuring its influence on subsequent agent behavior. MemoryDojo is easily extensible, consists of 155 multi-session long-horizon trajectories covering four domains, and supports five different memory systems. Through our evaluation, we find that all memory systems are susceptible to memory poisoning, with substantial variation in downstream effects. We further propose a memory-aware poison optimization process that increases the poisoning efficacy, highlighting differences between memory architectures. We hope that MemoryDojo will support research on understanding and mitigating agentic memory risks.