Learning to Unlearn: Machine Unlearning via Learning the Unlearning Behaviors
2026-08-17 • Machine Learning
Machine LearningArtificial Intelligence
AI summaryⓘ
The authors address the problem of removing specific data from trained machine learning models to respect privacy laws. Instead of designing complicated unlearning methods manually, they propose a new approach called Learning-to-UnLearn (L2UL) that learns how to unlearn data efficiently. Their experiments show that L2UL performs almost as well as retraining the model from scratch but is much faster, especially with large datasets and big models like ResNet. This makes the process of unlearning data more practical for real-world use.
machine unlearningprivacy legislationunlearning functionmodel retrainingLearning to Optimizemodel-agnosticResNetdata removalefficiencyscalability
Authors
Hang Zhang, Kaifeng Zhang, Yixiao Ma, Weijie Xu, Ye Zhu, Kai Ming Ting
Abstract
Various machine unlearning techniques have been developed in response to privacy legislation requirements, enabling individuals to exercise their legal right to have their data $D_f$ removed from a machine learning model. This process is typically accomplished via the use of an unlearning function denoted as $U$. Existing methods focus on designing an intricate $U$ to unlearn $D_f \subset D$ from a previous model $A(D)$, so that the unlearned model performs as closely as possible to the retrained model $A(D \setminus D_f)$. However, these methods often suffer from high computational costs when dealing with massive training data, as the complex structures of $U$ become a bottleneck even for models with fewer parameters. Inspired by Learning to Optimize, we introduce the first learning-based model-agnostic approach, Learning-to-UnLearn (L2UL). Our core insight is to shift from manually designing $U$ to learning the unlearning behaviors from a distribution perspective, thereby acquiring a simple and efficient $U$ via learning. Our experimental results demonstrate that the accuracy achieved by L2UL is comparable to that of retraining while exhibiting impressive efficiency, particularly in data-intensive scenarios. Furthermore, we validate the performance and scalability of our method on larger models ResNet.