Trustworthy virtual memory protects private data while aiding remote ai models
Semantic-TVM: Structure-Preserving Trustworthy Virtual Memory for Memory-Augmented and Tool-Using Agents
Artificial Intelligence
Summary
When AI systems use tools or look up memories, they sometimes expose private or sensitive information. The authors found that simply hiding all sensitive information can make the AI less useful or leak data in other ways. They created Trustworthy Virtual Memory (TVM), which keeps exact sensitive data safely local while sending only safe summaries to the remote AI. This approach keeps the useful context intact and helps AI complete tasks better without exposing private information.
What this means in practice
- •For healthcare data teams: Enable remote AI to process sensitive patient records without exposing exact private details, preserving utility and privacy simultaneously.
- •For enterprise ai platform developers: Build AI systems that interact with private data and external tools securely by locally masking sensitive parts while preserving useful context for trusted execution.
Authors
Yu Li, Qikun Cai, Tao Huang, Chen Hou
Abstract
Memory-augmented and tool-using agents expose exact private values when remote LLMs process retrieved memory, tool actions, and intermediate observations. One-way masking limits direct exposure but removes values needed for trusted execution and can leak them through later observations. We propose Trustworthy Virtual Memory (TVM), a closed-loop runtime that keeps exact-value state local while presenting a protected view to the remote model. Within this single runtime, Rule-TVM replaces whole protected fields with locally recoverable handles, and Semantic-TVM instead replaces only sensitive spans predicted by a trusted local model, preserving surrounding task-relevant context. On Memory-EHR and Memory-RAP across two providers, span-level projection recovers most of the EHR utility lost under whole-field replacement (Task Success 84.17% vs. 52.33% on DeepSeek) while measured exposure stays low and workflows remain executable.