What to Remember, What to Reveal: Privacy-Aware Memory for Conversational Agents
2026-08-17 • Cryptography and Security
Cryptography and Security
AI summaryⓘ
The authors created a new way for chatbots to remember user information while protecting privacy. Their system, called SP-Mem, keeps sensitive data separate and only shares it when really needed and allowed by the user. This approach helps chatbots personalize conversations better without risking unnecessary exposure of private details. They tested SP-Mem and found it balances privacy and usefulness well.
long-term memorypersonalized conversational agentsprivacy protectionpersonally identifiable information (PII)memory architecturedata sanitizationuser consentlarge language models (LLM)privacy benchmark
Authors
Wenjie Wang, Wenhe Si, Xinyue Xu, Yue Xu
Abstract
Long-term memory enables personalized conversational agents to retain user information across sessions. However, existing memory architectures primarily optimize for utility while neglecting the risks of unnecessarily storing and reusing private attributes such as personally identifiable information (PII). Addressing privacy risks in personalized memory is challenging because simply removing sensitive values can undermine system utility. Therefore, privacy protection for memory agents should govern the full life cycle of sensitive values rather than only sanitizing individual records. To address this gap, we introduce Sanitized Privacy-Mapped Memory (SP-Mem), a privacy-aware memory architecture that decouples memory utility from exact private-value exposure. SP-Mem provides a full life-cycle privacy design that identifies and separates sensitive information from raw user inputs, stores sanitized content and exact private values in isolated structures, and selectively retrieves private values based on task requirements and user consent. We further introduce a privacy-aware memory benchmark that jointly evaluates response quality, privacy behavior, and inference cost. Extensive experiments across multiple LLM-based agents show that SP-Mem achieves stronger personalization while reducing unnecessary privacy exposure. Code and data are available at https://github.com/Jensassss/SP-Mem.