Machine unlearning should focus on output control for safety
Risk-Aware Output Control Should be the Focus of Machine Unlearning
Machine Learning
Summary
Machine learning systems can sometimes produce harmful outcomes like biased or incorrect results, leading to the need for machine unlearning — a way to remove certain data from a trained model. The authors argue that instead of making an unlearned model look similar to one retrained from scratch, it is more important to control the model’s actual outputs with statistical guarantees. This means ensuring the model behaves safely and predictably in practice, which is more useful and reliable for real users. They support this idea with experiments and relevant theory.
What this means in practice
- •For ml deployment teams: Ensure that updates removing sensitive data come with certifiable control over model outputs rather than relying on retraining closeness.
- •For data privacy officers: Evaluate compliance by focusing on guaranteed changes in model output behavior instead of internal model similarity after data removal requests.
A position paper. It proposes an approach and reports no results.
Authors
Yahya Alkhatib, Yuyang Xiao, Wee Peng Tay
Abstract
Well-documented failure modes of machine learning (ML) systems, including bias, toxicity, hallucination, and leakage of sensitive information, has driven the rise of machine unlearning (MU). Most existing work, however, targets parameter-space indistinguishability, seeking unlearned models that are as close as possible to models retrained from scratch on retained data. In this paper, we argue that this objective is misaligned with practical and user-relevant concerns: MU should prioritize statistically guaranteed output-space control rather than parameter-space similarity. We contend that output control with statistical guarantees is the operationally meaningful notion of unlearning, as it supports rigorous and robust objectives, scales to modern deployment settings, and enables interpretable guarantees. We substantiate this position with empirical evidence and with recent theoretical results that, although not originally formulated in this framing, are consistent with it.