PiSAs: Benchmarking Contextual Integrity in Multi-User Agentic Systems
2026-07-06 • Multiagent Systems
Multiagent SystemsCryptography and Security
AI summaryⓘ
The authors discuss how large language model (LLM) agents are increasingly used within organizations, which creates new privacy risks because sensitive information can accidentally leak between users. Current privacy tests don't catch these risks well, so the authors created PiSAs, a new benchmark that checks if information sharing is appropriate and who should see it. Their tests show that even advanced models often fail to keep private information safe or limit access correctly. This suggests that more work is needed to protect privacy in shared AI systems.
Large Language ModelsPrivacy BenchmarkContextual IntegrityData SpillageAgentic SystemsInter-agent CommunicationShared MemoryInformation Access ControlPrivacy-preserving Strategies
Authors
Shubham Gupta, Nazanin Mohammadi Sepahvand, Abhinav Kumar, Cem Subakan, Spandana Gella, Pierre-André Noël, Perouz Taslakian, Eugene Bagdasarian, Valentina Zantedeschi
Abstract
As LLM agents evolve from single-user assistants into shared organizational infrastructure, new privacy risks emerge: inappropriate information may not only be exposed through outputs for external recipients, but also internally across users through inter-agent messages, shared memory and agents. These data spillage risks are not captured by existing privacy benchmarks grounded in contextual integrity (CI) as they focus primarily on either single-user settings or interactions between independently owned agents. We introducePiSAs (Privacy in Shared Agentic systems), a benchmark for assessing unintentional leaks with dual CI annotations: whether an information is appropriate for the task, and which users may legitimately access it. This enables direct measurement of cross-user spillage across agentic system components and interfaces, such as outputs, inter-agent communication, and memory. PiSAsis system-agnostic and supports evaluation across different agent topologies and memory regimes. We find that, although system design improves CI compliance, results are bottlenecked by incorrect LLM judgment calls: even state-of-the-art models fail to reliably filter inappropriate content or restrict transmission to authorized users. Our findings underscore the need for privacy-preserving strategies, beyond those studied in this work.