MemSentry detects attacks in AI systems with persistent memory
MemSentry: A Framework for Detecting Persistent Memory Poisoning in Agentic AI
Cryptography and Security
Summary
Agentic AI systems that remember information over time can be tricked by attackers who insert harmful content into their long-term memory, causing the AI to misbehave later. The authors created MemSentry, a system that carefully checks each request to add new information to AI memory and decides automatically whether to accept it, review it, or quarantine it based on trust and risk factors. They tested MemSentry using simulated scenarios and found it accurately detects dangerous memory changes, especially from outside attackers. For trusted insiders, it flags suspicious actions for human review rather than quarantining them automatically.
What this means in practice
- •For enterprise ai security teams: Detect and manage attempts to tamper with persistent memory in AI systems to maintain trusted and safe AI operation.
- •For cybersecurity platform developers: Integrate MemSentry’s framework to automatically screen AI memory updates for potential attacks and escalate insider risks for human review.
Authors
Ayan Roy, Kaustuvi Basu
Abstract
Agentic AI systems with persistent memory introduce a distinct attack surface known as memory poisoning, in which adversarially crafted content is stored in long-term memory and subsequently influences future agent behavior. Such attacks can suppress security alerts, facilitate privilege escalation, alter trust relationships, or override security policies without modifying the underlying model weights or system prompts. To address this threat, we present MemSentry, a formal, configuration-driven framework that intercepts proposed persistent-memory writes and produces deterministic Accept, Review, or Quarantine decisions. MemSentry evaluates each write by jointly considering source trust, semantic risk, attack radius over a component-dependency DAG, access risk, and a signed security-state delta that captures whether an operation weakens or strengthens the system's security posture. We instantiate the protected environment using a 20-asset random dependency DAG and a 10 x 20 user access-control matrix, and evaluate the framework over 1,000 GPT-4-generated scenarios using a stratified 70/30 train/test split. Semantic classification is treated as a pluggable component rather than a primary contribution, and we compare four representative approaches: rule-based Regex, TF-IDF+SVM, SBERT+LR, and SetFit. SBERT+LR achieves the best overall performance with 91.7% accuracy and a 0.908 macro-F1 score, while all four methods detect 100% of external quarantine-class threats. For verified insiders, where source trust is maximal (T = 1), MemSentry does not automatically quarantine suspicious operations but instead escalates potentially dangerous writes for human review, making semantic classification important for accurately capturing insider intent.