BatchNorm hidden effects can mislead machine unlearning tests
The BatchNorm Illusion: Diagnosing Normalization Artifacts in Machine Unlearning Evaluation
Machine Learning
Summary
Machine unlearning tries to erase specific data from an AI model without starting over, but testing if this works can be tricky. The researchers found that a common part of AI models called BatchNorm can give false signals, making it look like data is forgotten when it actually isn't. This happens because BatchNorm changes some internal settings just by looking at data again, confusing evaluation results. They showed that using another method called GroupNorm fixes this problem and that the false results were not caused by actual changes in the model's memory. Their work helps improve how scientists measure unlearning in AI.
Machine unlearningBatchNormNormalization artifactsFixed-point operatorForget accuracyLinear probeGroupNormMembership inference attackModel evaluationRunning statistics
Authors
Aaryaman Kalani, Murari Mandal, Dhruv Kumar, Mohan Kankanhalli, Yash Sinha
Abstract
Approximate machine unlearning aims to remove the influence of specific training data from a trained model without retraining from scratch. We identify a previously undocumented confound in how unlearning is evaluated on BatchNorm-based architectures: a single forward pass over retain data, an operation that modifies no weight, can deterministically rewrite the model's normalization state and reverse the apparent surface-metric forgetting. We formalize this operation as a weight-preserving fixed-point operator and prove that any pre-versus-post gap it induces is provably attributable to BN running statistics rather than to any modification the unlearning method made to the weights. This attribution claim cleanly separates measurement failure (BN artifact) from encoder failure (residual weight-encoded information, recently documented in concurrent work), and the same operator framework yields a unique decomposition of linear-probe elevation into BN-measurement-bias and encoder-geometry components. Empirically, the artifact reverses headline forget accuracy by up to 78 pp across nine evaluated methods on standard benchmarks; an attacker with as few as 10 unlabeled images recovers most of the masked accuracy; and a strict GroupNorm control reduces the artifact to zero across all methods. The tested membership-inference attacks change little under recalibration, locating the observed evaluation failure in forget accuracy and linear probing.