Video researcher improves long video understanding without training
VideoResearcher: Self-Improving Tool Design for Long-Video Understanding
Computer Vision and Pattern Recognition
Summary
Understanding long videos is hard and usually requires a lot of manual work to create tools that can analyze them well. The authors created VideoResearcher, a system that works like a human researcher by automatically designing, testing, and improving video analysis tools without needing extra training. It repeatedly improves its tools by looking at what it can’t do well and fixing those gaps. This method gets close to the quality of tools designed by humans but without the usual effort and time.
What this means in practice
- •For video analytics developers: Create automated systems that improve their video analysis tools over time for better long-video processing without retraining models.
- •For content moderation teams: Deploy adaptable video understanding tools that refine themselves to detect evolving patterns or issues in lengthy video content.
Authors
Dingqiang Ye, Dongdi Zhao, Kaishen Wang, Qingqiao Hu, Jingchen Sun, Yijun Liang, Yuqi Jia, Yiqiao Huang, Yunjie Tian, Jiaxing Zhang, Chuanyang Jin, Ke Zhang, Vishal M. Patel, Di Fu
Abstract
Video agents have made substantial progress in long-video understanding. Yet effective video-agent systems require costly, time-consuming manual design and trial and error. Current self-improvement methods either refine low-impact prompts, recombine predefined micro-tools, or struggle with convergence in harness optimization. To bridge this gap, we target high-impact video-tool with VideoResearcher, a training-free multi-agent framework that autonomously designs, tests, and refines tools for video understanding, like a human researcher. VideoResearcher operates through dual Solving and Evolving loops: it analyzes tool-use trajectories to identify capability gaps, coordinates specialized agents to develop and validate executable tools, and reuses evolved tools to strengthen evidence acquisition in subsequent video reasoning. Through iterative tool refinement and validation, it progressively strengthens evidence acquisition without updating model parameters. VideoResearcher achieves state-of-the-art performance among self-improving agents and approaches the human-designed upper bound, demonstrating a training-free paradigm for long-video understanding that expands agent capabilities through autonomous tool development while reducing costly manual engineering.