IntentQA: Intent Question Answering in Videos by Cognitive Context Reasoning
2026-08-24 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors introduce a new task called IntentQA to help computers understand the reasons behind people's actions in videos, not just what is happening. They created a large video question-answering dataset and designed tests to check how well models can handle tricky, changed examples. Their method, X-CaVIR, uses different kinds of helpful context like the situation in the video, contrasts between examples, and everyday knowledge to improve understanding. They also made their system's reasoning easier to follow by combining video captions with answers from a question-answering model. Experiments showed their approach outperforms others and stays reliable even with harder test cases.
Video Question Answering (VideoQA)Intent ReasoningContrast SetsLarge Language Models (LLMs)Contrastive LearningCommonsense ReasoningSituational ContextExplainable AICross-modal LearningBenchmark Evaluation
Authors
Jiapeng Li, Ping Wei, Wenjuan Han, Song-Chun Zhu, Lifeng Fan
Abstract
Video understanding requires intelligent agents to transcend mere recognition of visual facts and comprehend the underlying intents behind human actions (often termed the "dark matter" of social intelligence). To bridge the gap between visual observation and intent reasoning, we introduce a novel task, IntentQA, and contribute a large-scale VideoQA dataset specifically tailored for this purpose. However, recognizing that standard metrics may overestimate capabilities due to dataset biases, we go beyond simple accuracy to rigorously evaluate model robustness. We augment the benchmark by generating five distinct contrast sets via Large Language Models (LLMs) and introducing a "Contrast Performance Decline" metric. We propose the X-CaVIR (eXplainable Context-aware Video Intent Reasoning) framework, which leverages three types of "Cognitive Context" to enhance video analysis: i) Situational Context via a cross-modal Video Query Language (VQL) module, ii) Contrastive Context via a Contrastive Learning module, and iii) Commonsense Context via a Commonsense Reasoning module. Crucially, to overcome the opacity of traditional black-box models, we refine the integration of LLMs within X-CaVIR by employing a transparent pipeline that synergizes video captions with VQA model outputs. This approach not only improves performance by effectively utilizing rich commonsense knowledge but also renders the reasoning process explicitly interpretable. Extensive experiments demonstrate the effectiveness of our components, the superiority of X-CaVIR over state-of-the-art baselines, and its stability against perturbations on the contrast sets.