Summary
Machine unlearning means making a trained AI forget specific data it learned from. This paper studies a method called Deep Perturbation Learning (DPL), which changes training images and labels in specific ways to make the AI forget data. The authors find that DPL does not work well for making the AI forget data directly, nor does it consistently help keep the AI's performance good or speed up other forgetting methods. They also fix bugs in the original DPL code that affected results and show that previous claims about DPL’s power need careful re-examination. Their work applies only when deleting random data points and does not rule out other methods.
What this means in practice
- •For ai engineers: Avoid using influence-based perturbations alone as a reliable method to delete specific training images from AI models.
- •For security teams: Use the provided audit checklist to critically evaluate claims about data deletion methods before deployment in privacy-critical systems.
Tested on simulated data.
Authors
Chenkai Wu, Chrispine Kambimbi, Qinyang Zeng, Jun Yan
Abstract
We evaluate Deep Perturbation Learning (DPL), which perturbs training images and labels along influence-derived directions, in three roles in which prior work has positioned it for machine unlearning: a direct deletion signal (the strongest claim), a utility-preserving regularizer, and a warm start for adversarial unlearning. Evidence for the weaker roles has been used to support the stronger one, so we test each role separately under a matched protocol with exact-seed retraining baselines. An audit of the public implementation identifies two correctness issues: image directions are computed on augmented, normalized tensors but applied to raw images, and the label perturbation falls below float32 resolution, leaving labels unchanged. After correcting the image-perturbation pipeline, DPL fails the direct-deletion criterion on CIFAR-10/ResNet-18 in all three paired seeds. Its utility effects are inconsistent in sign across seeds, and once direction-computation time is counted it underperforms simple warm-start baselines. A one-seed Tiny ImageNet check likewise does not favor DPL as a regularizer or warm start; preprocessing inconsistencies in the released code make the direct comparison there inconclusive. These results cover random instance deletion only and do not rule out influence-based methods in other deletion regimes. We release a role-matched evaluation protocol and an audit checklist for perturbation-based deletion claims.