Unified multimodal models can leak private data via hidden watermarks
The Poisoned Conversation: Privacy-Leaking Watermarks in Unified Multimodal Models
Cryptography and SecurityComputer Vision and Pattern RecognitionMachine Learning
Summary
Multimodal AI models can understand and create text, images, and more all in one conversation. The authors find that sensitive details shared in a chat can secretly influence later images generated by the model, embedding hidden marks that reveal private information. This means even when users think their data stays private on their own devices, a manipulated model can leak secrets without obvious signs. The researchers demonstrate this risk and show it can be very accurate while keeping the model useful.
What this means in practice
- •For ai security teams: Detect and prevent hidden privacy leaks embedded via watermarks in multimodal AI systems.
- •For privacy compliance officers: Assess risks of multimodal models leaking confidential user data through unintended watermark channels.
Authors
Tobias Braun, Jonas Henry Grebe, Emil Sivic, Patrick Mohr Gordillo, Hossein Shakibania, Marcus Rohrbach, Anna Rohrbach
Abstract
Multimodal models are increasingly shifting toward unified architectures that understand and generate text, images, and other modalities within a shared conversational context. This design enables fluid interaction across modalities, but it also changes the privacy threat model: Information revealed in one part of a conversation may remain accessible when the model later generates content in another modality. This risk is particularly concerning in settings where users rely on locally deployed models for privacy, assuming that sensitive interactions remain confined to their device. We introduce Privacy-Leaking Watermarks (PLWs): invisible, trigger-dependent watermarks that a malicious model provider can condition on prior chat history. With this adversarial intervention, the usual separation breaks: a sensitive keyword or semantic cue mentioned earlier in the conversation can cause a later, unrelated image to carry a hidden yet detectable watermark. PLWs pose a novel threat to users of unified multimodal models: A poisoned model can retain utility while covertly turning image generation into a channel for privacy leakage, even when deployed locally. Across 13 sensitive-attribute triggers and two model families, PLWs reach up to 100.0% TPR at 1% FPR. For example, across all tested conversational separations, OmniGen2 detects every prior disclosure of depression while falsely flagging only 1% of images generated without such a disclosure.