Tracer detects erased concepts in diffusion models fast and accurately
Your Unlearning Gives You Away: Identifying Erased Concepts in Diffusion Models
Machine LearningArtificial Intelligence
Summary
Diffusion models can be unlearned to remove certain concepts, but people using these models may not reveal what was erased. The authors created Tracer, a tool that finds out which and how many concepts were removed, without needing to generate images or have prior knowledge. Tracer uses a lightweight analysis of the model’s weights, making it much faster and more precise than previous approaches. This helps identify changes in models that were supposed to forget specific information.
What this means in practice
- •For model deployment teams: Detect which concepts were removed in diffusion models to verify compliance and content restrictions quickly.
- •For digital content moderators: Identify erased image or video concepts in generative models to ensure harmful or sensitive content is properly unlearned.
Authors
Kaiyuan Deng, Yuchen Li, Yang Xiao, Bo Hui, Geng Yuan, Xiaolong Ma
Abstract
Existing attacks on unlearned diffusion models assume that the erased concepts are known in advance and focus on recovering them. In practice, however, model providers may not disclose which concepts have been removed, and even with access to the original base model, an adversary may still lack a clear target to attack. In this paper, we aim to answer the following critical but overlooked questions: which concepts have been erased from the model, and how many have been erased in total? To this end, we present Tracer, a framework that rapidly and accurately identifies erased concepts and estimates their number. Tracer efficiently identifies erased concepts without generating and classifying images. By combining lightweight spectral analysis of weight footprints, it enables efficient search over large candidate vocabularies. To distinguish multiple erased concepts, we introduce a footprint coverage objective that guides sequential discovery. Tracer estimates the number of erased concepts by detecting a sharp decline in candidate confidence as the selected concepts account for the erasure footprint, without requiring labeled examples for calibration. The framework requires only lightweight linear algebra and limited forward probes, with no prior knowledge of the unlearning algorithm. Experiments across text-to-image and text-to-video backbones and diverse unlearning methods demonstrate that Tracer identifies erased concepts and estimates their number in seconds, achieving 150 to 137,000 times and 133 to 20,000 times speedups over MIA and brute-force search on image and video models, respectively, with substantially higher identification accuracy.