Differentially private memory improves large language model context handling
DP-MIND: Differentially private memory-guided Long context Inference
Artificial Intelligence
Summary
Processing very long texts with large language models while keeping user information private is challenging. The authors developed a system called DP-MIND that breaks long inputs into smaller parts and remembers important details privately. Their method adds noise to protect privacy but uses smart learning to keep useful memories despite the noise. They also fix small errors caused by the noise without revealing private data. Tests show DP-MIND works better than previous methods for very long text tasks with privacy guarantees.
What this means in practice
- •For cloud ai service providers: Enable privacy-preserving processing of extremely long user texts in cloud-based AI services without exposing sensitive data.$Commercial implications: Allows hosting privacy-aware long-context LLM inference services that protect user confidential inputs at scale.
- •For enterprise ai engineers: Integrate private long-context inference capabilities into enterprise applications handling sensitive documents and communications.
Authors
Zhihao Liu, Yitian Cheng, Shuang Liu, Rui Wen, Yuke Hu, Zhan Qin
Abstract
LLM-as-a-Service has become a dominant paradigm for accessing large language models without requiring local deployment. However, the process raises severe privacy concerns when user queries may include sensitive or confidential contexts. Differential privacy (DP) provides a principled approach to protecting such inputs, but existing DP inference methods are primarily designed for short- or moderate-context settings and struggle with million-token-scale inputs due to limited context capacity and severe utility degradation under DP perturbation. In this paper, we propose DP-MIND, a memory-guided framework that scales differentially private inference to million-token contexts. DP-MIND enables private long-context inference by locally perturbing the input and processing it chunk by chunk with a compact memory that preserves task-relevant information across the sequence. To make this memory robust to DP noise, we optimize the memory construction policy directly on perturbed inputs using reinforcement learning, guided by both final answer correctness and memory consistency. We further recover perturbation-corrupted answer spans through a local denoising module without exposing the original context to the external model. We provide theoretical privacy analysis of DP-MIND and further conduct extensive empirical analysis across multiple long-context benchmarks with context lengths up to the million-token scale, demonstrating that DP-MIND consistently outperforms existing DP and long-context inference baselines in both scalability and utility.