LLM agents leak confidential data through combined document facts
Agent Mosaic: Recognizing Confidentiality Leaks in Agentic LLM Systems
Cryptography and SecurityArtificial IntelligenceComputation and Language
Summary
Large language model (LLM) agents can accidentally reveal secrets by combining bits of information from multiple documents, even if each piece alone seems safe. The authors tested several LLM agents with examples from cybersecurity, medicine, and military contexts and found that these agents sometimes created and disclosed sensitive conclusions without authorization. Attempts to limit this by adding clear labels or rules helped but did not fully prevent leaks. This shows that LLM agents need stronger safeguards when handling confidential data to avoid accidentally sharing private information.
What this means in practice
- •For enterprise security teams: Detect and reduce confidentiality leaks in LLM-based systems managing sensitive corporate data by monitoring agent answers for forbidden information combinations.
- •For health it administrators: Improve privacy controls on AI agents working with medical records to prevent accidental disclosure of protected health information through analysis of composite answers.
Authors
Daniel Ruiz, Luca Luceri, Emilio Ferrara
Abstract
Modern LLM agents can retrieve information from accessible documents, and then disclose a sensitive conclusion assembled from them. This mosaic effect creates a critical risk for agents operating over confidential data. We quantify its realization with 30 synthetic cybersecurity, medical, and military examples, yielding 10,800 final answers from three LLM agents in controlled simulation and 1,440 answers from live agents in two open-source harnesses. Even under benign queries designed to be answerable from one document, GPT-5.6-Sol, the best performing model tested, discloses forbidden composites in 7.08% of answers (95% CI: 3.75-10.67%) in simulation, with worse disclosure rates in live harnesses. Adding visible classification headers and explicit aggregation rules reduces pooled disclosure rates substantially, yet unsafe disclosures persist in 32.37% of simulated answers (95% CI: 28.33-36.19%) across all intents despite these safeguards. Furthermore, increased refusals do not guarantee confidentiality, since significant refusals reveal the prohibited conclusion regardless. These results point to a troubling conclusion: sensitive data leaks via mosaic-style events are an observed confidentiality failure mode in the evaluated settings at non-negligible rates. They motivate stronger controls on what agents may infer and disclose in sensitive data environments with mixed-clearance users.