Typed decision models often reveal private causes alongside results
TUPLE-Bench: Evaluating and Mitigating Privacy-Utility Trade-offs in Typed Decision Models
Cryptography and SecurityArtificial Intelligence
Summary
Many decision systems not only give a result but also show extra details like probabilities and confidence levels. The authors found that these extra details can accidentally reveal private information that was meant to be hidden. They tested a new method to reduce this privacy leak while keeping the system useful for its intended purpose. Their work shows it’s important to carefully check each part of a system’s output to protect sensitive information.
What this means in practice
- •For data privacy teams: Improve privacy audits by identifying fields in model outputs that leak sensitive cause information despite access controls.
- •For machine learning engineers: Apply workflow-constrained privacy quantization to reduce private data exposure while maintaining model performance on downstream tasks.
Authors
Yiqu Yang, Qiaoyuan Zheng, Yang Tian, Yu Fan, Elliott Ash
Abstract
Typed decision models often return more than a selected category. They also expose probabilities for each candidate and, sometimes, a separate confidence score. These fields support downstream decisions but may reveal information beyond a recipient's authorization. We introduce TUPLE-Bench (Typed decision models' Utility and Privacy Leakage Evaluation Benchmark), a benchmark of paired synthetic cases in which recipients may learn a coarse operational category but not its fine-grained cause. Across nine models, instructions to withhold the cause fail to protect it: 91.88%-99.93% of responses reveal the cause through direct selection or residual scores. Even when causes are excluded from the candidate set, full outputs enable 62.85%-75.35% recovery on held-out cases, compared with 51.04%-57.64% using only the selected category (50% chance). Full outputs also reduce mean downstream cost in all 27 model-task comparisons. However, confidence fields increase leakage without consistently reducing cost. We further introduce workflow-constrained privacy quantization (WCPQ), which reduces secret recovery while preserving the behavior and cost of specified downstream workflows. Our findings highlight the need to audit decision interfaces field by field and workflow by workflow.