Vision language models improve with smarter training on poor data
On-Policy Distillation for Vision-Language Model Adaptation, an Effective Paradigm on Low-Quality Multimodal Data
Computation and Language
Summary
Training smaller vision-language AI models often uses a bigger model to teach them, but this can be tricky when the data is low quality or different from usual examples. The authors propose a new approach where the teaching method changes based on how reliable each data example is. They use a controller that decides how much to trust the big model, prior knowledge, or hard labels for each sample during training. This way, the smaller model learns better even with noisy or varied data, improving its ability to adapt to new tasks.
What this means in practice
- •For computer vision developers: Improve adapting compact vision-language models to new or noisy datasets by using adaptive distillation targets during training.
- •For machine learning platform engineers: Integrate on-policy distillation controllers into training pipelines to boost transfer learning with minimal runtime cost.
Authors
Hongyuan Zhang, Xianda Guo, Yanlun Peng, Qianlong Yang, Yubin Guo, Pinhan Fu, Mulin Chen, Xiaozhen Qiao, Ping Luo
Abstract
Knowledge distillation offers an efficient route to transfer a task-adapted vision-language teacher to a compact student. The training target in current vision-language distillation methods is typically constructed from the teacher prediction and applied uniformly to all training samples, making it unreliable under class and domain shifts. In this paper, we argue that distillation target construction should be treated as a dynamic training decision rather than a fixed recipe. To this end, we propose OnPoKD, an on-policy distillation framework for vision-language model adaptation. To the best of our knowledge, OnPoKD is the first framework that applies on-policy distillation to vision-language model adaptation by learning target construction as a policy decision. OnPoKD learns a lightweight controller that constructs sample-wise adaptive targets using reliability and disagreement cues from the teacher model, student model, and zero-shot prior. Instead of relying on a fixed teacher prediction, the controller dynamically balances teacher supervision, zero-shot prior guidance, and hard-label anchoring through bounded policy actions, allowing the distillation target to adapt to varying sample reliability and training stages. The policy controller is updated with validation feedback, encouraging target construction to optimize transferability rather than merely fitting the training distribution. Since the controller is only used during training, OnPoKD can be seamlessly integrated into existing vision-language distillation pipelines while preserving the original inference architecture and test-time cost. Extensive experiments on Base-to-novel generalization and Cross-dataset transfer benchmarks show that OnPoKD consistently improves over strong vision-language distillation baselines.