Membership inference risks in on-policy self-distillation revealed
OP-Print: Tracing Membership Imprints in On-Policy Self-Distillation
Machine LearningCryptography and Security
Summary
When computer models teach themselves by using their own past predictions (on-policy self-distillation), they may unintentionally memorize who was in their training data. This paper shows that such self-teaching methods leave behind distinct patterns that can be detected by clever attacks, even when outsiders have limited information. The authors developed a new attack approach called OP-Print that finds these hidden memory clues more effectively than previous methods. Their results highlight potential privacy concerns in this teaching style.
What this means in practice
- •For machine learning engineers: Assess privacy risks when deploying self-distillation models by using OP-Print to detect if training data membership can be inferred.
- •For privacy compliance teams: Evaluate the likelihood of sensitive data leakage from models trained with on-policy self-distillation using the OP-Print attack framework.
Authors
Qi Li, Xinchao Wang
Abstract
On-policy distillation has been widely adopted for alleviating the distribution mismatch of off-policy distillation. Self-distillation further removes the need for an external teacher by using the student's own outputs, enhanced with extra context or demonstrations, as supervision. This teacher-free design makes on-policy self-distillation practical, yet its privacy risks remain largely unexplored. In particular, using two coupled views of the same training example may strengthen its memorization. Motivated by this, we present the first study of membership inference attacks against on-policy self-distillation. We investigate both gray-box and black-box threat models and propose a unified attack framework, named OP-Print. OP-Print builds on our finding that the supervision imprint in on-policy self-distillation is sparse and structured, concentrating on uncertainty-sensitive correction sites rather than across the entire rollout. Exploiting this property, we construct attack signals tailored to the two access levels: in the gray-box setting, OP-Print directly aggregates the privileged teacher-student residual correction along the same rollout; in the black-box setting, it measures restart-based output-basin stability as an observable surrogate for the hidden correction process. Across diverse models and distillation schemes, all baseline attacks remain essentially at random guessing even under gray-box access, whereas OP-Print is broadly validated to be effective in both gray-box and black-box settings.