Small AI model matches long video question accuracy with less computing power
Ambient @ EgoLongQA 2026: Distilling Long-Video perception into a Sub-2B Model
Computer Vision and Pattern Recognition
Summary
Understanding long videos is hard for small AI because they usually need big models. The authors made a clever system that learns from a bigger, smarter AI but itself is much smaller and faster. Their smaller AI can answer questions about ten-minute videos almost as well as the big one, using just over one percent of the big model's size. They also used a smart trick to reduce its size slightly without losing accuracy, making it eligible for a challenge with strict model size limits. This shows that smaller AI models can still do a good job by learning carefully from bigger models.
vision-language modelegocentric videomodel distillationparameter pruningmultiple-choice QAagentic pipelineembedding tablemodel sizeforward passwearable AI
Authors
Logesh Kumar Umapathi
Abstract
We describe our entry to the EgoLongQA track of the Wearable-AI Challenge in ECCV 2026, which placed first in the <=2B parameter division with 0.8279 on the held-out test set. Our system is a single 2B vision-language model that answers multiple-choice questions about ten-minute egocentric videos in one greedy forward pass; It is obtained by distilling the junior perception module of a tool-using agentic pipeline, not the agent itself into a small student, using teacher traces filtered to those that answered correctly. it reaches 89% of the accuracy of the large agentic pipeline using 1.1% of its parameters. This raises a 27.1% base model to 81.4% on our held-out questions. The 2B backbone has 2.2132B parameters and therefore over the divisional limit, to make the entry admissable we prune the multilingual embedding table from 248,320 to 143,469 rows, reaching 1.9985B with provably identical logits on retained rows.