HierDoc: Hierarchical Page-to-Region Evidence Routing for Long-Document Visual Question Answering
2026-07-31 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the problem of answering questions about multi-page documents by finding important information first at the page level, then at the smaller region level. They introduce HierDoc, a two-step system that first picks relevant pages and then selects key parts within those pages to help answer questions. Both steps are optimized separately but work together, allowing the model to keep a broad overview while also focusing on details. Their approach improves performance compared to previous methods, showing that combining coarse page selection with fine region selection is helpful.
Visual Question AnsweringMulti-page DocumentHierarchical Evidence RoutingSet PredictionPage SelectionRegion SelectionStructured-set RewardsOCRAnswer ModelLong Document Understanding
Authors
Rongjian Gu, Wengang Zhou, Junyu Xiong, Yonghui Wang, Bing Yin, Bei Wang, Houqiang Li
Abstract
Multi-page document visual question answering requires locating sparse evidence at both the page and region levels. Existing approaches typically emphasize one level over the other: page-centric methods focus on page acquisition, with region operations serving mainly as navigation aids, whereas region-centric methods assume that the relevant pages have already been supplied. Consequently, page and region selection remain disconnected rather than forming successive evidence decisions. We propose HierDoc, a hierarchical evidence-routing framework that formulates long-document evidence acquisition as two-stage set prediction from pages to regions. A page policy selects evidence pages from the full document; these pages are then parsed for semantic elements, after which a region policy selects the elements passed to a downstream answer model. Both answer-agnostic policies are optimized with stage-wise GRPO using granularity-specific structured-set rewards. The answer model receives selected full pages together with selected region crops and OCR or table text, preserving global context while emphasizing fine-grained evidence. Across the evaluated benchmarks, HierDoc achieves state-of-the-art or competitive performance among open-weight systems, improving LongDocURL by 16.87% relative to the strongest reported open-weight baseline. Controlled ablations further show that selected regional evidence improves the page-only system in accuracy and F1 by 5.51% and 4.82%, respectively. These results demonstrate the benefit of organizing coarse page routing and fine-grained region routing as successive, separately optimized stages of a unified evidence-acquisition process.