Llms save privacy budget by paying only for needed private data access
Only Pay What You Must Spend: On-Demand Privacy Budget Payment for Differentially Private RAG
Cryptography and SecurityDigital Libraries
Summary
Using large language models on sensitive data risks exposing private information. The authors show that current methods spend too much privacy allowance even on operations that don't need private data. They introduce SparsePay-RAG, which uses public data first and only 'pays' privacy costs when truly accessing private data. Their approach saves privacy budget and still provides useful responses under strict privacy settings.
What this means in practice
- •For data privacy engineers: Manage sensitive data use with controlled privacy spending when deploying large language models for data retrieval tasks.
- •For enterprise ai developers: Build privacy-aware LLM-based services that optimize privacy guarantees by minimizing unnecessary private data exposure.$Commercial implications: Enables creation of commercial AI tools that meet strict privacy rules while efficiently using private data.
Authors
Zhonghao Sun, Zhiliang Tian, Xinyue Fang, Shuo Ma, Juhua Zhang, Yiping Song, Dongsheng Li
Abstract
Deploying large language models (LLMs) on sensitive data via Retrieval-Augmented Generation (RAG) introduces severe privacy risks. Recent studies apply Differential Privacy (DP) to LLMs with RAG for formal privacy guarantees. However, existing DP-RAG frameworks rapidly exhaust the privacy budget. Although recent efforts attempt to save the budget by narrowing the retrieval scope or sparsifying private generation, these methods themselves cumulatively consume the budget, whereas they could actually rely merely on public information or at a negligible one-time privacy cost. This mismatch fails to align budget expenditure with the model's actual reliance on private data, causing substantial waste on operations that require no private access. To address this, we propose SparsePay-RAG, adopting "only pay what you must spend" as its core principle. Using public information as a zero-privacy prior, it charges the privacy budget only for the private increment. Specifically, SparsePay-RAG narrows the retrieval scope via public topic-guided clustering, adaptively controls private access frequency without privacy cost through isotonic cross-layer trajectory fitting, and compresses per-access budget via DP contrastive decoding. Under strong privacy constraints, experiments show SparsePay-RAG achieves superior privacy-utility trade-offs over baselines.