RouteDRAG improves multimodal document question answering accuracy
RouteDRAG: Learning to Route and Control Agentic Execution for Multimodal Document RAG
Artificial Intelligence
Summary
Answering questions from long documents with text, tables, and images can be tricky because it requires combining lots of information. The authors found that sometimes a step-by-step agent approach helps, but not always, and using it too much can slow things down or cause errors. They created RouteDRAG, a method that decides when and how to use this agent approach efficiently. This improves accuracy while cutting down unnecessary extra work in answering questions.
What this means in practice
- •For web application developers: Improve question answering accuracy in apps that process long documents combining text, tables, and images by selectively applying agentic reasoning.
- •For business analytics teams: Enhance automated report analysis by routing queries through efficient retrieval and reasoning paths to reduce computational overhead and errors.
Authors
Zekai Chen, Hao Zhu, Yu Liu, Xunkai Li, Zejian Kang, Rui Qian, Quanjiang Li, Rong-Hua Li
Abstract
Web-based applications increasingly rely on understanding long multimodal documents, where answering questions often requires combining information across pages, tables, and figures. Agentic retrieval-augmented generation addresses these tasks through iterative retrieval and reasoning. However, Agentic execution is not always necessary, introducing additional computation and potentially incorrect answers. Our empirical analysis reveals that Agentic execution provides query-dependent benefits, while additional operations may improve, preserve, or degrade answer accuracy. These findings motivate controlling both Agent invocation and subsequent execution. To address these challenges, we propose RouteDRAG, a learning-based framework comprising four components. Unified Document Index organizes multimodal content into a graph-backed index and derives query representations from initial retrieval. Adaptive Path Routing preserves exact answering and predicts the relative benefit of Agent over shorter execution paths. Dynamic Agent Execution maintains an evolving workspace and refines Planner proposals to control tool use and termination. Sequential Policy Learning first trains operation control from observed outcomes and then trains path routing using the resulting Agent's performance. Across two benchmarks, RouteDRAG achieves an average accuracy improvement of 3.47 percentage points over shorter-path-only execution while assigning only 79.3% of queries to Agent. Notably, it also surpasses uniform Agent execution on both benchmarks while requiring fewer answering calls. Our code is now available at https://github.com/Ereshkigal1/RouteDRAG.