Approximate unlearning method speeds up data removal from prototype classifiers
Approximate Unlearning in Prototype-based-classifiers
Machine Learning
Summary
Sometimes, specific data needs to be removed from machine learning models, such as when users request deletion or when data is found to be incorrect. Retraining a model fully can be very slow, so this paper proposes a faster way to remove certain data's effects from a specific type of model called prototype-based classifiers. The authors show their method is almost as accurate as retraining the whole model but works much faster across several datasets. Their approach can handle different sizes and types of data for removal and does not need changes to how the original model is trained.
What this means in practice
- •For data privacy teams: Remove the influence of specific user data quickly from prototype-based classification models in compliance with privacy regulations.
- •For machine learning engineers: Accelerate correction of model errors caused by noisy or corrupt training data by approximately unlearning selected samples without retraining from scratch.
Authors
Sreejita Ghosh, Mohammad Mohammadi
Abstract
The recent data and AI regulations and governance needs have highlighted the need for removing specific data even from models trained on or with them. While retraining the entire model is a theoretically trivial solution, it is computationally expensive, thereby motivating research in Machine Unlearning. This paper introduces a robust approximate unlearning technique for the family of prototype-based classifiers called Generalized Learning Vector Quantization, aimed at removal of effects of specified samples, either due to deletion request or due to them being identified as noisy or corrupt. We first show the impact of the different sizes of the forget set, as well as the characteristics of the data in this set, on the unlearned model. To address the sources of instabilities detected in this unlearning method we propose an adapted version of this technique, such that the unlearning is robust to the characteristics of the forget set. Thereafter using the intrinsic interpretability property of GLVQ and appropriate metrics defined in unlearning literature, we show the efficacy (approximately 0 deviation of parameters and performance of unlearned model relative to retrained model) and efficiency (6X for smallest dataset to 10^(5)X in the largest dataset) of the proposed unlearning techniques compared to naive retrained, on six benchmark datasets for tabular data recommended by literature. Although in this work we investigated only binary class problems, the proposed robust unlearning strategy can be readily applied for removing sample effects from any trained GLVQ model, i.e., it does not require any modification in the training scheme of GLVQ.