Continual unlearning improves data removal in diffusion models
Continual Data Unlearning in Diffusion Models via Transition-based Regularization
Machine Learning
Summary
Sometimes, AI models need to forget specific training examples without losing what they’ve learned overall. This paper shows how to help diffusion models forget requested data one step at a time without breaking earlier forget requests or making the model worse at creating images. The authors save small snapshots of changes made during each forget action and use these to guide future forgets, preventing them from undoing past work. Their method keeps memory needs low and balances both forgetting and preserving quality better than existing approaches.
What this means in practice
- •For ai model maintainers: Remove specific data points from diffusion models over time without losing earlier deletions or image quality.
- •For privacy compliance teams: Support sequential data deletion requests in AI systems while maintaining overall generative capabilities.
Authors
Sunbeom Jeong, Sehwan Kim, Sangwoo Hong, Jungwoo Lee
Abstract
Data unlearning in diffusion models aims to remove the influence of specific training examples without suppressing the broader concepts they represent. However, when deletion requests arrive sequentially, updates for new requests can degrade generative utility and undermine earlier deletions. We propose a continual data unlearning framework that uses completed deletion transitions as directional references to regularize future updates. For each request, we record changes in denoiser responses on the same fixed noisy inputs before and after unlearning. Rather than matching full post-deletion responses, we apply a one-sided penalty that discourages reversal along the recorded directions relative to the post-deletion references, while leaving orthogonal response changes and progress beyond these references unpenalized. To keep storage independent of the number of requests, we maintain a fixed-capacity bank of representative transition records. Records are selected based on the local sensitivity of progress along their recorded directions to parameter updates, allowing them to be retained even when their penalties are inactive. Empirical evaluations show that the proposed framework achieves a better balance between deletion persistence and generative utility than existing unlearning baselines as requests accumulate, using only a small transition memory.