DeCoRAG: Cognitive Decoupling and Semantic-Aware Cropping for Complex Document Understanding
2026-07-27 • Information Retrieval
Information RetrievalComputer Vision and Pattern Recognition
AI summaryⓘ
The authors studied how to better understand complex documents that include both pictures and text using multimodal Retrieval-Augmented Generation (RAG). They found that current methods relying on Vision-Language Models (VLMs) struggle when trying to focus on small visual details within dense layouts, causing big errors they call a 'Visual Attention Sink.' To fix this, the authors developed DeCoRAG, which first creates a broad semantic anchor to guide focus and then prunes irrelevant parts, making the system more accurate and efficient. Their approach improved accuracy in tests and reduced the amount of data needed for processing.
Multimodal Retrieval-Augmented GenerationVision-Language ModelsGraph RAGVisual Attention SinkSemantic AnchorRegion-Aware PruningDocVQAToken OverheadGraph ConstructionSemantic Clusters
Authors
Shuo Wang, Kai Zhang, Wenyuan Huang, Yizheng Yu, Xia Liao, Junming Su, Qing Wang, Fang Xi
Abstract
Advancing multimodal retrieval-augmented generation (RAG) for complex document understanding presents a formidable dual dilemma of accuracy and efficiency, particularly in graph RAG. Processing structurally sparse yet visually dense layouts, such as extracting a tiny data marker from a financial chart, often incurs computationally prohibitive token overhead while still triggering catastrophic hallucination. However, multimodal Graph RAG pipelines rely on graph-construction stages that assume Vision-Language Models (VLMs) can resolve sparse semantics within high-density layouts. We challenge this assumption, revealing that forcing VLMs to localize visual evidence, interpret semantics, and extract relations triggers a "Visual Attention Sink," a mechanism driving catastrophic semantic loss, while full-page processing incurs massive computational overhead. Controlled interventions verify that this failure is boundary-driven rather than content-specific and that semantic anchoring mitigates it. To fundamentally correct this flawed paradigm, we introduce DeCoRAG, a multimodal Graph RAG pipeline that shifts knowledge processing from coupled visual-semantic reasoning to "Cognitive Decoupling." Rather than passively processing raw pixels, its graph-construction stage establishes a macroscopic Semantic Anchor to neutralize the attention sink. This anchor subsequently drives our Region-Aware Pruning and Cropping (RAP-Crop) mechanism, shifting the reasoning space from dense, noisy backgrounds to purified, intent-driven semantic clusters. The resulting graph supports hybrid retrieval and answer generation. Across complex document benchmarks, DeCoRAG improves the semantic pass rate by up to 12.5 percentage points over the strongest baseline and generalizes to DocVQA. RAP-Crop reduces offline graph-construction prompt tokens by 40.8% without sacrificing end-to-end accuracy.