Attack method extracts images from multimodal retrieval systems
Walking the Embedding Space: Datastore Extraction from Multimodal RAG
Cryptography and SecurityArtificial Intelligence
Summary
Some AI systems retrieve images to help answer questions, but they can accidentally reveal private or sensitive images. The authors developed a way to trick these systems into leaking many images by carefully choosing inputs that combine known and unknown pictures. This method works without knowing the system’s inner details and recovers hundreds of images after a few thousand tries. Their findings show the need for better protections in AI systems that handle images and text together.
What this means in practice
- •For security teams: Identify and mitigate risks of image data leakage in AI systems combining text and image retrieval.
- •For legal compliance officers: Assess privacy vulnerabilities in AI assistants that return visual data to ensure data protection standards are met.
Authors
Maria Carmen Jica, Ali Satvaty, Suzan Verberne, Fatih Turkmen
Abstract
Multimodal Retrieval-Augmented Generation (MRAG) has emerged as a reliable and cost-effective technique of grounding the generative capabilities of Multimodal Large Language Models (MLLMs) into relevant, up-to-date, external knowledge. Despite presenting several benefits, such as reducing hallucinatory behavior, they also introduce new attack surfaces, including leakage of private information and vulnerabilities against data extraction attacks. In this paper, we introduce $\immrag$, an adaptive and automatic data extraction attack procedure operating in a black box setting against \emph{image-returning} MRAG, a configuration in which the retrieved visual artifact is itself the response. Each query blends an attacker-held shadow image with an image already recovered from the system, and relevance-weighted resampling steers subsequent queries towards regions of the embedding space that still yield novel retrievals. Unlike current extraction attacks that aim to persuade the model towards data leakage by placing a malicious query as a textual prompt, $\immrag$ embeds the malicious instructions inside a user-given input image. We evaluate $\immrag$ on three plausible and distinct real-world scenarios: medical assistant, document-focused helper and general purpose tool. The experiments involve the study of the effectiveness of the attack on multiple CLIP-family retrievers, as well as the impact of various generators. A single 2500-query run reconstructs up to 611 distinct radiology images, 566 document scans and 416 general-purpose images under local-feature correspondence, and reaches up to $5.6\times$ as many distinct datastore items as a non-adaptive baseline. Our results show the urgent need for safeguards specifically designed for multimodal data.