Privacy conductor enforces document access policies outside language models
Privacy Conductor: An Architectural Enforcement of Contextual Integrity in LLM-Based Document Retrieval
Cryptography and Security
Summary
Deciding who can see certain documents is important for privacy, especially when using AI language models. These models can tell what’s in a document but struggle to decide who should get access to it. The authors created Privacy Conductor, a system that uses clear rules set by people to control access, instead of letting the AI decide. Their tests show that relying on AI alone can lead to mistakes in who sees what, so it’s safer to keep policy decisions outside the AI.
What this means in practice
- •For enterprise security teams: Enforce privacy policies on internal document access by using a rule-based gate that prevents sharing sensitive data via language model searches.
- •For healthcare it administrators: Ensure compliance with patient privacy laws by separating document sensitivity labeling from AI retrieval to reduce accidental disclosures.
Authors
Ryan Thornton, Mir Mehedi A. Pritom, Maanak Gupta, Jessee Roberts
Abstract
Large language model (LLM) that retrieve internal documents within an organization must decide who may receive them. Existing approaches based on Contextual Integrity ask the model to make that decision, so access depends on probabilistic reasoning. We present Privacy Conductor, an architecture that removes the decision from the model: a deterministic gate enforces a policy written by an authority, and the model receives only what the requester may already see. On a synthetic clinical corpus of 2,520 requester--document flows, the gate reproduces its policy exactly, so the remaining risk lies in the document labels it reads. As label source is of primary importance, we compare LLM and human generated labels. Across seven configurations of six models from two model families, LLM-written labels degrade the policy in every case. Models assign a document's sensitivity tier correctly on 78--86% of documents, but reproduce its access-control list on as few as 23%. Relabeling the same documents five times with an identical request changes 13 to 124 access decisions per configuration, a result that does not depend on our own labels. On 214 flows from HIPAA court cases, the most accurate configuration (87.8%) still permits 11 of the 20 disclosures the law prohibits. LLMs can describe what a document contains, but not decide who should receive it. Hence, we recommend privacy policy should be authored by accountable personnel and enforced outside the model.