Text-to-image models can selectively erase concepts without retraining
Compositional Concept Erasure in Text-to-Image Diffusion Models via Hierarchically Grounded Semantic Surgery
Computer Vision and Pattern Recognition
Summary
It can be important to remove certain concepts from AI image generators, like copyrighted or unsafe items, without changing the entire model. The authors show a new way to do this that works well even when the concepts appear inside complex descriptions. Their method figures out exactly which parts of the text relate to the concept to erase and keeps other details intact, all without retraining the AI. This improves how well the undesired concepts are removed while preserving the rest of the image generation.
What this means in practice
- •For ai model deployment teams: Remove copyrighted or unsafe concepts from deployed text-to-image models without retraining or modifying model weights.
- •For content moderation engineers: Prevent generation of undesired concept instances in image outputs by more precisely erasing concepts in compositional text prompts.
Authors
Chen Dai, Ganyu Zou, Nathan Self, Kevin Piper, Ramachandra Rao Seethiraju, Karthik Shyamsunder, Chang-Tien Lu, Naren Ramakrishnan
Abstract
Removing copyrighted, unsafe, or user-specified concepts from a deployed text-to-image diffusion model is now a practical requirement. Weight-editing methods can suppress fixed targets, but they require per-target retraining and modify the model checkpoint. Training-free methods, on the other hand, are deployment-friendly, but they suffer from text-side routing failures on compositional prompts. In such prompts, the erase target may be invoked through a related class rather than its lexical name, and its modifiers may migrate onto preserved objects. This paper proposes Hierarchically Grounded Semantic Surgery (HGSS), a training-free framework for compositional concept erasure. The framework lifts both the routing signal and the edit operator used by text-side erasure. First, hierarchical span grounding resolves erase-target spans through lexical, taxonomic, and semantic evidence, while guarding against broad-hypernym and compound-head false positives. Second, dynamic attribute binding refines the text conditioning during early denoising via a counterfactual reference and a preserve-aware cross-attention objective, keeping surviving attribute-noun bindings intact. HGSS selectively removes the erase target without updating model weights or adding learned parameters. On SEE, HGSS cuts hierarchical evasion from 29.54 to 10.02 and roughly halves pairwise attribute leakage, achieving the best Neighbor E and AttrP scores among the reported erasure methods. On UnlearnCanvas, HGSS slightly improves the six-metric average over the matched Semantic Surgery baseline, reaching state-of-the-art.