Graph model unlearning improved by stepwise sample removal strategy
CUNO: Curriculum and Preference Optimization for Stable Graph Unlearning under Mass Deletion
Machine LearningArtificial Intelligence
Summary
When graph-based AI models need to forget some of their training data, they usually perform much worse if a lot of data is removed all at once. The authors found this happens because not all deleted nodes are equally important, but existing methods treat them the same. They propose a new method called CUNO that removes data gradually, starting with easier-to-forget parts, and uses a special technique to keep the model working well on the remaining data. Their experiments show this approach keeps more useful performance, especially when a large portion of data is deleted.
graph unlearningmass deletioncatastrophic unlearningcurriculum learningstructural dependenciesnegative preference optimizationmodel utilityforget setmachine learninggraph models
Authors
Chenhan Zhang, Ali Braytee, Madhushi Bandara, Xin Hao, Paul J. Kennedy, Massimo Piccardi, Raymond Owen
Abstract
Graph unlearning removes the influence of designated training data from a trained graph model without retraining from scratch. However, existing methods suffer a sharp drop in model utility under large deletion ratios (mass deletion), a phenomenon we refer to as catastrophic unlearning. We find that a key cause is the uniform treatment of all deleted samples, which is particularly damaging in graph learning: structural dependencies cause different nodes to play vastly different roles in the learned model, yet existing methods apply the same forgetting operation to the entire forget set. Based on this insight, we propose CUNO, a curriculum-based graph unlearning framework that removes the forget set progressively, ordering samples by their estimated unlearning difficulty across multiple stages. CUNO further employs a distribution-level negative preference optimization (NPO) objective at each curriculum stage that steers the model away from its original behavior on the current forget subset while preserving retained performance. Our theoretical analysis shows that the curriculum design is most beneficial when the forget set spans a wide range of unlearning difficulty, a condition naturally satisfied under mass deletion. Comprehensive experiments confirm that CUNO consistently mitigates catastrophic unlearning: at 20% deletion, it retains 74% of the original utility compared to 26-53% for existing methods, and maintains more than half the original utility even at 50% deletion. Our code is publicly available at https://anonymous.4open.science/r/cuno-D4FF.