Vision language models struggle to forget individuals fully
What Does It Mean to Forget a Person? Individual-Level Unlearning in Vision-Language Models
Cryptography and SecurityComputers and Society
Summary
It’s hard for AI models that understand pictures and words together to forget specific people completely because their information is mixed up across images and text. The authors created a new way to test how well these models can forget individuals, not just simple traits. They found that even when a model forgets some details, it can still recognize or recreate information about a person. This shows that forgetting a person is more complicated than just erasing certain facts.
What this means in practice
- •For privacy engineers: Improve techniques to remove individual identities from AI models that use both images and text to protect privacy.
- •For ai ethics compliance teams: Evaluate and certify AI systems’ ability to forget user identities across multiple data types for compliance with data protection laws.
Authors
Xiongtao Sun, Hui Li, Tiantong Wu, Jiaming Zhang, Fuyao Zhang, Wen Jun Tan
Abstract
Erasing individual identities from Vision-Language Models (VLMs) is uniquely challenging because personal data is entangled across modalities rather than stored as isolated attributes. However, existing multimodal unlearning benchmarks primarily evaluate attribute-centric forgetting, overlooking the more critical objective of individual-level unlearning: eliminating a model's ability to access, link, and reconstruct target-related information across modalities. To address this gap, we propose IDUnlearn-Bench, the first benchmark for individual-level multimodal unlearning in VLMs. It represents each individual as connected multimodal evidence and evaluates four task families: attribute access, identity access, identity binding, and identity reconstruction. Experiments on representative VLMs and unlearning methods show that successful attribute-centric forgetting often leaves substantial identity-level knowledge intact and can be non-monotonic: reducing one form of risk may amplify another. Models may suppress selected information while still identifying the target, linking records, or reconstructing the individual. These findings reveal a fundamental gap between forgetting information about a person and forgetting the person as a whole.