How Hard Can It Be? Hardness-Aware Multi-Objective Unlearning
2026-06-01 • Machine Learning
Machine LearningArtificial Intelligence
AI summaryⓘ
The authors focus on machine unlearning, which is about removing certain data's influence from a trained model while keeping the model useful for other data. They find that how hard this is depends on how similar the data to forget is to the data to keep. Their new method, called HAMU, uses this idea to improve forgetting specific data with less harm to the remaining knowledge. HAMU also warns when it’s impossible to improve forgetting without hurting performance. They tested HAMU on images and text, showing it works better than previous methods.
machine unlearningforget training dataretain dataconstrained optimizationmodel weights updatenon-convex modelsforget qualityretain utilityparallelizable algorithms
Authors
Jiangwei Chen, Xinyuan Niu, Rachael Hwee Ling Sim, Zhengyuan Liu, Nancy F. Chen, Bryan Kian Hsiang Low
Abstract
Machine unlearning aims to remove the influence of specific forget training data due to privacy, copyright or bias concerns while maintaining the model performance on the remaining retain data. Existing unlearning algorithms, such as optimizing a weighted combination of losses, have tried to achieve these objectives of improving forget quality and maintaining retain utility. However, they do not guarantee that these objectives can be improved by a specified extent for all forget and retain data. In this work, we address this limitation with a novel and theoretically-grounded approach from a constrained optimization perspective. Firstly, we identify that the hardness of reconciling both objectives can be quantified by the similarity between the forget data and the retain data. Next, we derive an unlearning algorithm (HAMU) with the overall goal of guaranteeing a specified improvement in forget quality while minimizing the retain utility cost/degradation by updating the model weights based on our hardness measure. Our hardness measure also informs users when retain utility degradation is unavoidable, i.e., both objectives cannot be improved simultaneously, and stopping should be considered. Our algorithm is applicable to non-convex models and is easily parallelizable, making it readily deployable in real-world scenarios. We empirically demonstrate HAMU's superior performance over baselines on both image and text datasets using large models. Our code is available at https://github.com/aoi3142/HAMU.