STORM: Internalized Modeling for Spatial-Temporal Reasoning in Video-Language Models
2026-05-25 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionComputation and Language
AI summaryⓘ
The authors introduce STORMS, a new way for large vision-language models (LVLMs) to understand videos by thinking through hidden patterns instead of writing out detailed reasoning in text. Their method trains the model in two steps: first, it learns to connect internal signals with video actions, and then it practices answering questions without showing its work step-by-step. This approach lets the model reason about changes over time in videos more quickly and simply during use, without needing extra tools or repeated video processing. Tests on several video reasoning tasks show STORMS is both more accurate and faster than previous methods.
vision-language modelsvideo reasoningchain-of-thoughtlatent trajectoriestemporal orderthought-video representationsanswer-only supervisioninference latencyvideo understandingreasoning pipelines
Authors
Yiming Liang, Yixiao Chen, Yiyang Zhou, Yixuan Wang, Shoubin Yu, Andong Deng, Fuxiao Liu, Qin Zhang, Chen Chen, Mohit Bansal, Huaxiu Yao
Abstract
Many video reasoning tasks require tracking motion, temporal order, and evolving visual states across frames. Existing methods built on large vision-language models (LVLMs) often address this challenge by externalizing reasoning through textual chain-of-thought (CoT), keyframe selection, repeated frame reinsertion, or external tool use. While effective, such pipelines increase inference-time latency and engineering complexity, and they force temporal-visual evidence to be serialized into text or repeatedly re-encoded from frames. Inspired by the intuition that visual reasoning can occur implicitly before verbalization, we propose STORMS (Spatial-Temporal reasOning via inteRnalized Modeling), a two-stage framework that teaches LVLMs to reason through bounded continuous latent trajectories instead of explicit textual CoT. In Stage I, STORMS aligns latent tokens with thought-video representations derived from generated videos, grounding the latent states in dynamic visual evidence. In Stage II, the model is further trained with answer-only supervision, encouraging the reasoning process to be internalized without step-by-step annotations. Generated thought videos are used only during training; at inference, STORMS performs a bounded latent rollout without regenerating videos, reinserting frames, or invoking external visual tools. Experiments on VideoMME, MVBench, TempCompass, and MMVU show that STORMS improves video reasoning accuracy while substantially reducing inference overhead compared with tool or video-generation-based reasoning pipelines.