InsightVQA: High-Dimensional Emotion-Cognitive Visual Question Answering Benchmark
2026-06-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors created InsightVQA, a big dataset with lots of pictures and questions to help computers better understand emotions in images. Unlike past work that just asks what emotion is shown, their questions also explore why those emotions happen and what might come next. They organized questions into three levels: spotting the emotion, understanding what caused it, and predicting responses based on reasoning. They also made a test set called InsightVQA-Bench and a baseline model named InsightNet to measure progress. Their results show that this dataset is tough for current AI to solve fully.
visual question answeringemotion recognitioncognitive reasoningvalencemultimodal large language modelsdataset curationhierarchical annotationtrigger extractionresponse intent prediction
Authors
Shiyu Wang, Ziyu Liu, Chaoyi Yu, Yujie Yin, Zhongqian Mao, Jing Chen, Jiaqi Song, Yunshi Lan, Yan Wang
Abstract
Visual emotion understanding requires models not only to recognize emotional states, but also to why they arise and perform higher-level cognitive reasoning. However, existing benchmarks mainly focus on emotion recognition, offering limited support for grounded understanding and response-oriented analysis. To address this gap, we introduce \textbf{InsightVQA}, a large-scale dataset for hierarchical visual question answering on emotion understanding and cognitive reasoning. Building from 351K images collected from six public sources, we apply a rigorous multi-stage filtering pipeline to curate 138K high-confidence images. Each image is annotated at three hierarchical levels: perception QA for emotion and valence recognition, grounded understanding QA constructed from visual trigger extraction through constraint-guided generation, and cognition QA centered on response intent prediction and sequential insight reasoning. In total, InsightVQA contains 725K QA pairs. We further present \textbf{InsightVQA-Bench}, a high-quality evaluation benchmark comprising 30K samples for fine-grained evaluation. To support evaluation, we introduce \textbf{InsightNet}, an emotion-tuned baseline for MLLMs. Results demonstrate that InsightVQA poses significant challenges for grounded emotion understanding and reasoning.