Large vision language models struggle to respect personalized privacy rules
Specify Privacy Yourself: Assessing Inference-Time Personalized Privacy Preservation Ability of Large Vision-Language Models
Computer Vision and Pattern RecognitionArtificial IntelligenceComputation and LanguageComputers and SocietyMachine Learning
Summary
Large vision-language models (LVLMs) can understand images and text together, but they can also reveal private information when asked. This paper looks at whether these models can follow personalized privacy rules that users give at the time of use. The authors created a big test set with many images and privacy instructions, then checked how well 21 top LVLMs protect privacy. They found most models perform poorly, but they also developed a new way to train models that greatly improves privacy protection.
What this means in practice
- •For ai product developers: Build vision-language AI systems that dynamically respect user-specified privacy constraints during image interpretation.$Commercial implications: Enables safer AI products that comply with personalized privacy demands, making them market-ready for sensitive consumer applications.
- •For data protection teams: Use new privacy alignment techniques to audit and improve privacy compliance of visual AI tools handling personalized data.
Authors
Xingqi Wang, Xiaoyuan Yi, Xing Xie, Jia Jia
Abstract
Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities but raise significant privacy concerns due to their abilities to infer sensitive personal information from images with high precision. While current LVLMs are relatively well aligned to protect universal privacy, e.g., credit card data, we argue that privacy is inherently personalized and context-dependent. This work pivots towards a novel task: can LVLMs achieve Inference-Time Personalized Privacy Protection (ITP$^3$), allowing users to dynamically specify privacy boundaries through language specifications? To this end, we present SPY-Bench, the first systematic assessment of ITP$^3$ ability, which comprises (1) 32,700 unique samples with image-question pairs and personalized privacy instructions across 67 categories and 24 real-world scenarios, and (2) novel metrics grounded in user specifications and context awareness. Benchmarking the ITP$^3$ ability of 21 SOTA LVLMs, we reveal that: (i) most models, even the top-performing o4-mini, perform poorly, with only $\sim$24% compliance accuracy; (ii) they show quite limited contextual privacy understanding capability. Therefore, we implemented initial ITP$^3$ alignment methods, including a novel Noise Contrastive Alignment variant which achieves 96.88% accuracy while maintaining reasonable general performance. These results mark an initial step towards the ethical deployment of more controllable LVLMs. Code and data are available at https://github.com/achernarwang/specify-privacy-yourself