Diffusion models enable verifiable unlearning of specific concepts internally
No Concept Escapes the Audit: Auditing-Aware Unlearning for Verifiable Concept Erasure in Diffusion Models
Machine LearningArtificial Intelligence
Summary
Text-to-image AI models sometimes create unwanted or harmful images, so people want to remove those ideas from the models. The authors found that past methods only change the model’s text input but don’t fully erase what the model has learned, meaning removed concepts can still be detected inside. To fix this, they propose AVCE, a new way to directly clean the model’s internal brain signals related to unwanted concepts. Their tests show AVCE removes these concepts more reliably while keeping the AI’s ability to create good images.
What this means in practice
- •For ai developers: Improve text-to-image models by reliably removing unwanted concepts without harming image quality.
- •For content moderators: Verify that prohibited content and styles have been effectively erased from AI image generation models' internal data.
Authors
Kaiyuan Deng, Yuchen Li, Gen Li, Yang Xiao, Geng Yuan, Xiaoyong Yuan, Bo Hui, Xiaolong Ma
Abstract
Text-to-image diffusion models can generate prohibited content, which motivates concept erasure through machine unlearning. Most erasure methods intervene at the text interface, through prompt modification or localized updates to text-conditioning weights, and they are evaluated by what the model outputs for given prompts. Such evaluation cannot see what the network still encodes. Latent-space auditing, which bypasses text conditioning and probes the denoising network directly, shows that erased concepts remain recoverable from internal representations. We find that this also holds for methods built to be robust against adversarial prompts, and that the problem grows with the number of erased concepts. We propose Auditing-Aware Unlearning for Verifiable Concept Erasure in Diffusion Models (AVCE), a framework that grounds erasure in the model's latent representations. AVCE audits the embedding neighborhood of each concept and condenses the discovered vulnerable directions into an anchor at the weakest geometric point. It edits cross-attention and self-attention projections in closed form at this anchor, then fine-tunes the two pathways with pathway-level auditing losses, using orthogonal gradient projection to consolidate multiple concepts. Experiments on SD v1.5, SDXL, and Flux 1.0 across object, explicit-content, and artistic-style unlearning show that AVCE reduces attack success rates by 5.07x and improves auditing scores by 3.84x over the strongest baseline, while preserving competitive generation quality.