BatchNorm hidden actions can mislead machine unlearning tests
The BatchNorm Illusion: Diagnosing Normalization Artifacts in Machine Unlearning Evaluation
Machine Learning
Summary
When a computer model is taught to forget certain training data, the usual tests to check if it actually forgot can be tricked. This happens because Batch Normalization layers inside the model update their internal settings even when the model’s main parts don’t change. The authors explain this hidden change creates a false impression that the model remembers more than it should, which earlier tests missed. They also show how this error can be fixed by using another method called Group Normalization. This helps make sure tests properly measure if the model truly forgets the data.
What this means in practice
- •For machine learning engineers: Improve evaluation of data removal methods by accounting for BatchNorm artifacts that distort forgetting accuracy.
- •For privacy compliance teams: Use more reliable metrics to verify that machine unlearning techniques genuinely remove data influence to meet privacy regulations.
Authors
Aaryaman Kalani, Murari Mandal, Dhruv Kumar, Mohan Kankanhalli, Yash Sinha
Abstract
Approximate machine unlearning aims to remove the influence of specific training data from a trained model without retraining from scratch. We identify a previously undocumented confound in how unlearning is evaluated on BatchNorm-based architectures: a single forward pass over retain data, an operation that modifies no weight, can deterministically rewrite the model's normalization state and reverse the apparent surface-metric forgetting. We formalize this operation as a weight-preserving fixed-point operator and prove that any pre-versus-post gap it induces is provably attributable to BN running statistics rather than to any modification the unlearning method made to the weights. This attribution claim cleanly separates measurement failure (BN artifact) from encoder failure (residual weight-encoded information, recently documented in concurrent work), and the same operator framework yields a unique decomposition of linear-probe elevation into BN-measurement-bias and encoder-geometry components. Empirically, the artifact reverses headline forget accuracy by up to 78 pp across nine evaluated methods on standard benchmarks; an attacker with as few as 10 unlabeled images recovers most of the masked accuracy; and a strict GroupNorm control reduces the artifact to zero across all methods. The tested membership-inference attacks change little under recalibration, locating the observed evaluation failure in forget accuracy and linear probing.