Benchmark tests AI agents on long complex research with images
Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents
Artificial Intelligence
Summary
It can be hard for AI systems to do really long research tasks where each step depends on the previous ones and some information comes from pictures or videos. The authors created a new test called Mr.LHDR that challenges AI agents to solve big research questions that need many steps and different types of info, like charts or maps. They found that current AI systems still struggle, especially when the tasks get longer or when images are involved, showing there's more work to do for deep, step-by-step reasoning with mixed info.
What this means in practice
- •For ai system developers: Assess and improve AI research agents' ability to handle extended, multi-step reasoning tasks with mixed text and image data.
- •For knowledge management teams: Evaluate tools that support complex investigation processes requiring integrating diverse document types and visual data.
Authors
Minghao Guo, Meng Cao, Sui Zhao, Siyu Ning, Xin Wang, Haoze Zhao, Jiaxuan Yang, Haihong Hao, Mingfei Han, Shunlin Rong, Haijun Wu, Xiaodan Liang, Xiaojun Chang
Abstract
Deep research agents are increasingly capable of web search, tool use, multimodal evidence analysis, and information synthesis. However, existing benchmarks mainly evaluate medium-horizon exploration and rarely test whether agents can sustain long, dependency-heavy research processes. We introduce Mr.LHDR (Multimodal real-world Long-Horizon Deep Research), a benchmark for evaluating real-world deep research over long, irreducible chains of interdependent evidence across eight categories. Each question is constructed from a hidden Node-Relation graph and requires an average of 12.1 necessary intermediate conclusions with a mean dependency depth of 10.4 before reaching a short, unique, and verifiable answer. Questions incorporate multimodal evidence, including images, maps, PDFs, logos, charts, tables, and video frames, with at least one non-text element that changes the reasoning state. Mr.LHDR evaluates both final answers and the correctness of intermediate conclusions under annotated dependencies. We evaluate general models, deep research systems, and agent frameworks using Overall Accuracy (OA), Strict Accuracy (SA), Checklist Score (CS), and Dependency-Aware Checklist Score (DACS). Results show that even the strongest system achieves only 43.1% OA and 34.3% SA, indicating that final-answer accuracy substantially overestimates complete research success. Removing images reduces DACS by 12.6 points, demonstrating the importance of multimodal evidence, while SA consistently declines as reasoning chains become longer. These findings reveal sustained, dependency-consistent evidence integration, rather than isolated fact retrieval, as a key bottleneck for current deep research agents.