Small video language models learn better reasoning with new training trick
Efficient Reasoning Distillation: Small Video-Language Models via Synthetic CoT and Difficulty-Aware Fine-Tuning
Machine LearningArtificial IntelligenceComputer Vision and Pattern Recognition
Summary
Video question answering models that understand videos and answer questions usually need to be very big to reason well. The authors show a way to teach a smaller model to reason almost as well by fine-tuning it with a few carefully chosen examples that include step-by-step explanations generated by a larger model. They found that putting these explanations after the answer instead of before works better for small models. This method is fast and uses less computing power, making smarter video language models easier to deploy on mobile and edge devices.
What this means in practice
- •For mobile app developers: Create video QA apps that run efficiently on phones by using smaller models trained with this reasoning distillation technique.$Commercial implications: This enables production of smarter mobile video understanding tools that are compact and fast, appealing to app developers needing deployable AI.
- •For video content platform teams: Use compact models fine-tuned with synthetic reasoning to improve automated video content understanding and question answering at scale with low compute costs.
Authors
Mantek Singh, Jeshwanth Challagundla, Siddharth Raina, Jasmin Jarsania
Abstract
We present an efficient method to distill reasoning capabilities into compact video-language models (VLMs) for video question answering (VideoQA). Our approach fine-tunes a 2B-parameter model using only $\sim$900 uncertainty-selected examples, each augmented with synthetic chain-of-thought (CoT) rationales generated by a 4B teacher. Despite its minimal compute cost - under two hours on a single A100 GPU - our method enables the 2B model to outperform VLMs up to 4$\times$ larger, and generalize across CinePile, ActivityNet-QA, and MLVU, approaching the performance of its own 4B teacher. A key finding is that placing CoT rationales after the answer - contrary to standard prompting - substantially improves reasoning in compact models. This insight challenges prevailing CoT conventions and reveals new alignment strategies under limited model capacity. Our findings offer a practical blueprint for training deployable, reasoning-rich VLMs suited for mobile and edge applications.