Training free timing method improves streaming video question answering
PACE: Plan, Attend, Check, and Emit for Training-Free Response Timing in Streaming Video Understanding
Computer Vision and Pattern Recognition
Summary
It can be hard for computer systems to know not just what to say about a video, but when to say it, especially while the video is still playing. The authors noticed common mistakes happen when answers come too soon or aren't stable as new parts of the video appear. They designed a new method called PACE that plans which pieces of the video to watch, checks if answers are reliable, and waits for the right moment to respond, all without extra training. Tests showed PACE made better-timed answers than previous ways across different video question models.
What this means in practice
- •For video analytics developers: Improve real-time systems that answer questions about live videos by more reliably deciding when to produce final answers without extra training.
- •For surveillance system engineers: Enhance automated video monitoring tools to avoid premature alerts by verifying events are complete before signaling.
Authors
Siqi Guo, Yuxia Chen, Wenbo Zhu, Yangguang Ji, Zhenxiang Jiang, Teng Zhong, Yu Zhang
Abstract
Streaming video understanding requires a model to decide not only what to answer but also when to answer. Evidence arrives progressively, and recent benchmarks identify premature responses, issued before the queried event has concluded, as a recurring failure mode. Existing approaches often learn response timing through additional training, while training-free alternatives typically rely on scalar gating signals that do not explicitly represent when an answer becomes final. We organize chunk-wise streaming errors into three patterns: sampling variation can make an answer unstable, a stable answer may lack support in the observed video, or an answer may be supported by the current evidence while the event remains unfinished. We propose PACE (Plan, Attend, Check, and Emit), a training-free framework designed around these three failure modes. Given a question, PACE drafts a plan specifying what to observe and what will signal completion. As the stream unfolds, it invokes an off-the-shelf video LLM at informative moments, evaluates each draft for stability, grounding in its cited frames, and completion, and records verified observations in an event ledger. Experiments on ProactiveVideoQA and OVO-Bench Forward show that PACE improves over the official protocols by up to 5.9 average PAUC points and 6.2 average accuracy points across proprietary and open-source backbones, yielding state-of-the-art performance.