Fisher conditioned subspaces improve on policy self distillation scores
SCOPE-OPSD: Fisher-Conditioned Privileged Subspaces for On-Policy Self-Distillation
Machine LearningArtificial Intelligence
Summary
The paper explores a way to improve on-policy self-distillation, a method where an AI learns from its own generated answers to get better at tasks like language modeling. The authors introduced a technique that uses a special mathematical projection based on something called the Fisher information to capture helpful signals from the differences between the AI's predictions and a teacher's guidance. Their approach shows consistent improvements across multiple large language models without adding extra computation during training or evaluation. This suggests a more efficient way to teach AI systems to learn from themselves faster.
What this means in practice
- •For language model engineers: Enhance training methods by incorporating Fisher-conditioned privileged subspaces to improve student-teacher alignment without adding inference cost.
- •For natural language processing developers: Improve AI model quality in short training budgets by using structured residual projections during model self-distillation.
Authors
Yunmeng Chen, Kunyu Wang, Peihan Li, Yi Wang, Shuyin Xia, Yi Liu, Xinyong Cheng, Dehui Wang, Xiangyong Zhai, Yanxing Liu, Song Liu
Abstract
On-policy self-distillation (OPSD) scores student-generated prefixes with a solution-conditioned self-teacher, yet transfers supervision only through next-token probabilities. We ask whether the aligned final-layer discrepancy offers a useful second channel, and how to test that channel without confusing its geometry with auxiliary strength. SCOPE-OPSD projects the privileged teacher-student residual onto a frozen rank-64 factor estimated from residual covariance and language-model-head Fisher sensitivity. It reuses the forwards already required by OPSD and adds neither rollouts nor inference-time modules. A matched Random control preserves the structured factor's rank and nonzero spectrum and uses per-arm gradient-RMS calibration, isolating the effect of the data-dependent orientation. Across the complete 25/50/75/100-step trajectories for Qwen3-1.7B, 4B, and 8B, Structured is never below Pure OPSD, with strict gains in 11 of the 12 model-checkpoint combinations and an exact tie at 4B step 25. Structured also exceeds matched Random in 10 of the 12 combinations. At step 75 on Qwen3-1.7B, Structured exceeds matched Random by 1.39 Macro Avg@12 points in each of two independent training reruns. A cross-fitted diagnostic also shows 4.40 times greater held-out privileged-gap capture than the matched random orientation. The results support a compact, Fisher-conditioned privileged subspace for short-budget OPSD.