On the Plasticity Collapse in Continual Machine Unlearning
Machine LearningArtificial Intelligence
Summary
The authors study how machine unlearning, which lets AI models forget specific training data, works over many repeated requests instead of just once. They discover a problem called 'plasticity collapse,' where the model gradually loses its ability to forget new data properly and may even start remembering old data again. Their experiments show this is a widespread issue across different models and datasets. This work highlights an important challenge for making unlearning reliable over time and suggests the need for new methods that keep the model flexible.
Authors
Yingdan Shi, Xiang Xu, Kaize Ding, Alfred O. Hero, Ren Wang
Abstract
Machine unlearning enables deep neural networks to selectively remove the influence of specific data in response to privacy and regulatory requirements. While prior work largely studies single-shot unlearning, real-world systems must accommodate continual unlearning, where multiple unlearning requests occur sequentially over time. In this work, we identify a fundamental limitation of this setting: plasticity collapse, a progressive breakdown in a model's ability to effectively forget. Through theoretical analysis of continual unlearning dynamics, we show that continual unlearning operations accumulate geometric constraints in parameter space, leading to saturated subspaces that restrict future updates. This structural effect induces two distinct failure modes: (1) Forward failure -- diminishing forgetting quality for subsequent tasks, and (2) Backward failure -- spontaneous re-memorization of previously forgotten information. Extensive experiments across multiple architectures, datasets, and methods in image classification confirm that plasticity collapse is not an artifact of specific implementations, but a pervasive phenomenon inherent to continual unlearning. Our findings reveal a critical barrier to the long-term reliability of machine unlearning systems and motivate the development of plasticity-preserving unlearning algorithms. Our code is available at https://github.com/TIML-Group/Continual-Machine-Unlearning-Plasticity-Collapse