Video based research agents create interactive reports linking moments
Beyond Watching and Searching: Benchmarking Agents for Video-centered Deep Research Reports
Computer Vision and Pattern RecognitionComputation and Language
Summary
Watching videos to gather detailed information is hard because you often miss important parts or how evidence relates. The authors designed a new task where AI agents watch a video and use other online info to answer questions, then create interactive reports that link to specific video parts. They also built a big test set and scoring method to see how well these agents do. Current AI systems still struggle with understanding all content deeply and citing sources accurately.
What this means in practice
- •For video content analysts: Generate detailed interactive reports from videos with linked relevant moments and supporting evidence to enhance video understanding.
- •For customer support teams: Provide interactive guidance reports based on product demo videos combined with external knowledge to help troubleshoot and verify procedures.
Authors
Jiahao Meng, Qize Yang, Yuxuan Wang, Yunfei Chu, Wensi Dai, Jin Xu, Yunhai Tong
Abstract
Deep research agents synthesize open-web evidence into long reports, but current benchmarks do not evaluate research centered on a source video. We introduce Video-Centered Deep Research Report Generation. Given a video and user request, an agent identifies research questions, retrieves external multimodal evidence, and produces an interactive report linked to relevant video moments. This task has direct practical value because it helps users locate relevant moments, verify claims, reproduce demonstrated procedures, and make informed decisions from video content. We present a skill-based workflow and OmniDR-Bench for this task. For evaluation, we propose Omni Atomic Rubric Trees (Omni-ARTs). Each tree contains three dimensions, 23 shared directions, and task-specific atomic binary criteria. The directions test audiovisual grounding, multimodal evidence use, and interactive presentation. Audited Rubric Construction (ARC) builds these trees through independent evidence collection, automated checks, behavioral calibration, and iterative human review. OmniDR-Bench contains 50 Chinese and English tasks across 19 realworld subcategories. It includes 6,414 criteria. Across 18 agent configurations, the best system scores 63.3 out of 100. Current agents still need broader content coverage, stronger causal and logical analysis across modalities, and more accurate citation support. We further analyze models, harnesses, skill configurations, video categories, and research trajectories. The code and data are released at https://github.com/marinero4972/OmniDR-Bench.