Self learning method improves language model answers by measuring information flow
Know Thyself, Teach Thyself: Internal Information Flow for Selective Self-Distillation
Machine Learning
Summary
When a computer program teaches itself to improve, it has to figure out which parts of its own knowledge help it learn better. The researchers introduced a way to track how information moves inside the program to decide what it should focus on learning next. They do this by seeing how much the program’s confidence changes after it recalls related knowledge. If the change is big enough, the program uses that example to teach itself. This approach worked well across several language models and topics, making self-learning more selective and effective.
What this means in practice
- •For language model developers: Improve model self-training by selectively using examples that cause meaningful belief changes in the model's answers.
- •For ai system engineers: Design self-improving AI systems that measure internal information changes to guide what data the model learns from next.
Authors
Rui Wang, Ruijie Wang, Bo Chen, Jiangxuan Long, Yingyu Liang
Abstract
Self-distillation turns knowledge distillation into a closed learning loop and offers a path toward recursive self-improvement. Without an external teacher, however, the model must determine both what information can improve its supervision and which induced changes should be learned. Existing methods typically improve teacher-generated data or select training examples in isolation, leaving the information transferred between these stages unmeasured. We introduce InFlow, a retrieval-guided on-policy self-distillation framework that models this process as potential-to-realized information flow. InFlow first retrieves potentially informative sources using certainty-calibrated hidden-state trajectories, then measures their realized effect through the Jensen--Shannon divergence between the teacher's initial and retrieval-conditioned answer beliefs. Examples with larger belief shifts are selected for on-policy distillation. Our analysis formalizes the information optimized by retrieval and selection and relates the answer-level shift to the teacher--student distillation gap. Across four open-weight language models and three knowledge domains, InFlow achieves the strongest cross-model average among the compared selection methods, with ablations supporting both stages of the framework. Our code is available at https://github.com/1240148048/INFLOW.