Materials machine unlearning improves by understanding deletion effects
Bounding Retraining Equivalence and the Deletion Floor in Materials Machine Unlearning
Machine Learning
Summary
When a specific data record is removed from a materials-focused machine learning model, related data can keep predictions accurate, making it hard to measure the impact of the removal. The authors define a new baseline called the deletion floor, which represents the expected prediction error after retraining without that record. They mathematically and experimentally show how similar retained data and local agreement affect this baseline and prediction changes. Their findings help clarify how well the model 'forgets' deleted records and suggest reporting multiple metrics to evaluate unlearning properly.
What this means in practice
- •For materials scientists: Improve reliability of machine learning models by assessing true effects after removing specific training data.
- •For machine learning engineers: Design more precise unlearning evaluation protocols for production models dealing with related or redundant data points.
Authors
Can Polat, Mustafa Kurban, Erchin Serpedin, Hasan Kurban
Abstract
In materials machine learning, closely related retained structures can sustain accurate property predictions even after removing a specific record, rendering post-deletion prediction error an ambiguous metric for machine unlearning. To resolve this ambiguity, we define the deletion floor as the expected target loss under a specified retraining procedure at the deleted request. Standard indistinguishability constraints yield a sharp interval bounding an update's target loss around this baseline reference. Theoretically, a conditional neighbor bound links a low deletion floor directly to retained fit, prediction regularity, and local label agreement, while an exact ridge identity isolates residual fit from the prediction change induced by record deletion. Empirically, controlled redundancy sweeps show an $\approx 8\times$ drop in median normalized retraining loss when one retained relative remains after deletion. Across two distinct fitting regimes in a paired Materials Project study, the lower-floor regime also exhibits a larger prediction change on more than 50% of the shared requests. Systematic comparisons against approximate updates and the original model decouple deliberate target suppression from preserved overall model utility. Consequently, request-level unlearning evaluations should report reference loss, prediction change, and retained utility together, interpreting post-deletion accuracy against what retraining itself leaves behind.