Multimodal attack fools ai safety checkers for harmful content
Mirage: Targeted Jailbreaks Against Post-Generation Safety Checkers in Multimodal Diffusion Models
Cryptography and Security
Summary
AI art tools use safety systems to stop harmful or inappropriate images from being created. This paper shows that these safety systems can be tricked by hiding unsafe content within harmless images in a way that the AI doesn't detect. The technique, called Mirage, works without needing to know how the safety system is built and can bypass different safety tools. It achieves this by manipulating how the AI understands images and text together.
What this means in practice
- •For ai content safety teams: Improve safety systems by testing defenses against attacks that hide unsafe content in safe images to bypass filters.
- •For security analysts: Identify vulnerabilities in AI-generated content filters by simulating attacks that do not require internal system knowledge.
Authors
Shanghao Shi, Chaoyu Zhang, Heng Jin, Hao Li, Ning Zhang, Y. Thomas Hou, Wenjing Lou
Abstract
Post-generation safety checkers are widely employed in diffusion-model pipelines to prevent unsafe generated content from reaching users. In this paper, we show that these safety checkers can still be bypassed by our proposed multimodal jailbreak attack, Mirage. Given an adversarial target, which can be either a reference NSFW image or a malicious text description, Mirage conceals the target within an imperceptible perturbation applied to a benign input image. Mirage exploits the shared representation space of multimodal encoders to jointly align the perturbed image with the cross-modal adversarial target while repelling its representation from commonly filtered NSFW concepts. Such semantic separation can transfer across representation spaces, enabling Mirage to evade heterogeneous safety checkers without accessing their internal details. We further employ lightweight black-box query refinement to improve evasion near the unknown safety boundary. We implemented Mirage on Stable Diffusion 1.5, Stable Diffusion XL, and Kandinsky 2.2, and evaluated it on two cross-modal NSFW datasets, covering both text-to-image and image-to-image generation scenarios. The experimental results demonstrate that Mirage achieves strong targeted jailbreak performance across diverse post-generation safety checkers with limited black-box queries while preserving high semantic alignment with the adversarial targets.