Summary
Agentic AI systems keep memories that influence their future decisions, but attackers can sneak bad information into this memory to cause problems without changing the AI itself. To stop this, the researchers created MemSentry, which carefully checks every memory change and decides if it should be allowed, reviewed by a human, or blocked. They tested MemSentry using simulated complex scenarios and various methods to understand the content’s risk, finding that one approach (SBERT+LR) worked best. This system helps catch dangerous changes from outsiders automatically, while suspicious changes from trusted insiders are flagged for human review to keep AI trustworthy.
agentic AIpersistent memorymemory poisoningsecurity policiessemantic classificationdependency graphaccess controlhuman reviewmachine learning classifierssecurity posture
Abstract
Agentic AI systems with persistent memory introduce a distinct attack surface known as memory poisoning, in which adversarially crafted content is stored in long-term memory and subsequently influences future agent behavior. Such attacks can suppress security alerts, facilitate privilege escalation, alter trust relationships, or override security policies without modifying the underlying model weights or system prompts. To address this threat, we present MemSentry, a formal, configuration-driven framework that intercepts proposed persistent-memory writes and produces deterministic Accept, Review, or Quarantine decisions. MemSentry evaluates each write by jointly considering source trust, semantic risk, attack radius over a component-dependency DAG, access risk, and a signed security-state delta that captures whether an operation weakens or strengthens the system's security posture. We instantiate the protected environment using a 20-asset random dependency DAG and a 10 x 20 user access-control matrix, and evaluate the framework over 1,000 GPT-4-generated scenarios using a stratified 70/30 train/test split. Semantic classification is treated as a pluggable component rather than a primary contribution, and we compare four representative approaches: rule-based Regex, TF-IDF+SVM, SBERT+LR, and SetFit. SBERT+LR achieves the best overall performance with 91.7% accuracy and a 0.908 macro-F1 score, while all four methods detect 100% of external quarantine-class threats. For verified insiders, where source trust is maximal (T = 1), MemSentry does not automatically quarantine suspicious operations but instead escalates potentially dangerous writes for human review, making semantic classification important for accurately capturing insider intent.