Long video agents choose when to stop gathering information efficiently
CASE: Cost-Aware Stopping for Efficient Long-Video Agents
Artificial IntelligenceComputer Vision and Pattern Recognition
Summary
It can be hard for computer programs that watch long videos to know when they've seen enough to answer questions. The authors created CASE, a method that helps these programs decide the right time to stop watching by weighing the cost of watching more versus how much better their answer might get. CASE learns from past examples to make these stop-or-continue decisions quickly. Tests show that CASE saves a lot of computational effort without losing accuracy, and it works well on new video tasks without extra training.
What this means in practice
- •For video analysis engineers: Reduce computation time and resource use in long-video question-answering systems by deciding when to stop processing without losing accuracy.
- •For multimedia platform developers: Improve user experience in video understanding applications by integrating a stop decision model that avoids unnecessary video processing costs.
Authors
Yiming Du, Chenghao Liu, Zhiyuan Liu, Fangxing Zheng, Zhao Wang, Junnan Nie, Songfang Huang
Abstract
Long-video agents can actively gather question-relevant evidence, but they typically leave a central decision implicit: when has the agent seen enough to answer? We propose CASE, a plug-in termination framework that frames this decision as policy-conditioned sequential stopping. At each causal checkpoint, CASE combines an auxiliary multiple-choice assessment of accumulated evidence with the host agent's execution state. From complete native trajectories, we construct a cost-aware target that compares answering now with stopping later along the same search path, accounting jointly for answer correctness and the full cost of continued reasoning. A lightweight Ridge regressor learns this decision gap and produces STOP/CONTINUE decisions. We evaluate three vision-language models with VideoSeek and AVP. On Video-MME, end-to-end accuracy changes by +0.67 percentage points on average while CASE reduces model-token use by 53.63%. The same frozen policies then transfer zero-shot to LongVideoBench and MLVU, with end-to-end accuracy changes of +3.38 and +4.58 points while saving 58.78% and 51.28% of model tokens, respectively. Across all agent-model-benchmark combinations, CASE attains the highest accuracy-efficiency Pareto-frontier coverage among the compared stopping methods (83.3%) at the selected operating points. Online execution preserves this favorable accuracy-efficiency trade-off and additionally reduces measured runtime by 54.1% on average. CASE provides a plug-in termination framework for long-video reasoning agents, enabling them to decide when further evidence acquisition is no longer worthwhile.