LLM agents often leak sensitive data despite strong task skills
AgentCIScope: Grounded Contextual Integrity Evaluation for LLM Agents
Artificial IntelligenceCryptography and Security
Summary
Large language model (LLM) agents are helping with tasks by using apps like email and calendars, but they often share sensitive information incorrectly. The paper presents AgentCIScope, a system that tests these agents on tasks with clear privacy rules based on real policies and personal data types. The authors tested 12 agents and found that even agents good at finishing tasks frequently leaked private information, especially on certain data types and apps. This tool helps understand and improve how these AI helpers keep private info safe.
What this means in practice
- •For enterprise ai developers: Evaluate and improve LLM agents’ adherence to privacy constraints across workplace applications using a unified test platform covering realistic scenarios.
- •For privacy compliance teams: Assess and monitor the risk of sensitive data leakage by AI agents handling user information in email, calendar, and messaging environments.
Authors
Chengquan Guo, Zhaorun Chen, Zinan Lin, Xiaodong Liu, Bo Li
Abstract
LLM agents are becoming increasingly capable of handling daily workspace tasks by interacting with application environments through tool calls. However, as agents read and write sensitive user data across email, documents, and other applications, a central privacy question is whether they preserve contextual integrity (CI), that is, whether the information flows they produce comply with context-specific disclosure constraints. Existing work either focuses on model-level privacy leakage or evaluates agents in limited application environments with manually constructed disclosure constraints. In this work, we introduce AgentCIScope, a unified platform and benchmark for evaluating agent-level contextual integrity through executable, multi-step tasks across diverse, realistic application environments. AgentCIScope transforms public and synthetic privacy cases into executable agent tasks, with disclosure constraints grounded in three criteria: governing privacy policies, persona-based agreement, and a predefined sensitive-attribute taxonomy. Based on AgentCIScope, we evaluate 12 agents on 936 tasks and find that strong task-completion capability does not guarantee contextual-integrity preservation: single-run leakage rates range from 32.5% to 94.4%, while repeating each task five times further increases leakage by 4-19 percentage points. Leakage varies substantially across attributes: personal preferences and booking details have average leakage rates of 84.7% and 76.5%, compared with 17.1% for credit-card information and 29.4% for SSNs; environments: calendar has the highest average leakage rate as an information source, while WhatsApp ranks highest as an information destination; and governing privacy policies. AgentCIScope provides deep insights into agent privacy leakage and serves as a unified testbed for developing privacy-preserving agents.