Papers for

document ai developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Vision language models struggle to use text even when they recognize it

VLMs Read Aloud: The Gap Between Recognizing and Using Visual Text

Abstract: Visual-text compression renders long contexts as images and is engineered around how much of the text a vision-language model (VLM) can recover. Yet recovered text is not usable text. VLMs can transcribe the text in an image word for word and still fail to answer from it; given back their own transcript, they answer nearly as well as from the original text. We call this difference the read-use gap. Recognizing and using visual text have different resolution requirements: Qwen3-VL-8B's recognition saturates at 15-pixel characters while use catches up only at 24 pixels, and scaling from 2B to 32B raises recognition without narrowing the gap. Putting the evidence into words closes it: thinking VLMs recite the evidence in their thinking, and their own transcript, written without seeing the question, recovers most of what thinking adds. In a preregistered held-out test, the gap ranks how much eight VLMs gain from their own words over pixels (Spearman $ρ$ = 0.86), which recognition rates do not. We propose ALOUD, which spends visual resolution on finding and recognizing evidence and language tokens on using it. Without keeping the source text, it doubles the accuracy of a frozen Qwen3-VL-8B on long documents compressed 5x and 10x: at 5x, from 27.7% to 54.8%, near the original text's 58.5%, with a context smaller than the compressed pages themselves. For less than the cost of reading the original text, it leads at equal budget every method that does without it, at every level, and beats AGAR and VERA, which paste the original text back; at 5x it also beats LensVLM-9B, which is trained and keeps the original text. Visual-text compression is limited not by how much text a model can recover, but by how much it can use.

Sun 1 NovComputer Vision and Pattern RecognitionComputation and Language
The gist
Vision-language models (VLMs) can read text in images perfectly but often fail to use that text effectively to answer questions. The authors discovered a gap between recognizing (reading) and using (understanding) visual text, which gets smaller when VLMs recite evidence in their own words. They propose a method called ALOUD that improves accuracy by focusing more visual detail on finding evidence and using language tokens to process it, nearly matching the accuracy of models that get the original text directly. This work shows that simply recovering text is not enough for good performance; using it properly is the real challenge.
Open → 2611.01517v1

New system evaluates explainability methods for document AI models

DocXEval: A Systematic Evaluation of Explainability in Multimodal Document Understanding

Abstract: Multimodal transformer models have driven unprecedented performance breakthroughs in modern document understanding. However, the inherent black-box nature of these models remains a major obstacle to their safe deployment in high-stakes sectors. Despite significant progress in explainability for vision and language tasks, explainable AI (XAI) research tailored to multimodal document understanding has received little attention. To address this gap, we present \ourframework{}, the first comprehensive explainability framework that adapts existing XAI methods and evaluation metrics to the multimodal (text, layout, and image) document understanding context, while introducing efficient multi-target adaptations of both methods and metrics for domain-specific tasks such as semantic entity recognition. Using \ourframework{}, we systematically evaluate 13 XAI methods across 3 document understanding tasks, 8 datasets, 4 models, and 17 evaluation metrics, and find that only a few methods perform consistently across all evaluation criteria, with Occlusion emerging as the most faithful and interpretable method overall. Through a dedicated meta-evaluation, we observe moderate to strong agreement among several faithfulness and complexity metrics while identifying certain ones that produce unreliable results. Finally, through multiple dataset-wide diagnostic analyses, we uncover key insights into model behavior, including the contribution of individual modalities across tasks and labels, spatial attribution patterns around target entities, and potential dataset biases. Overall, this work serves as both a benchmark and a practical guide for future XAI research in multimodal document understanding. To support reproducibility, we publicly release our code at https://github.com/saifullah3396/docxeval/.

Tue 27 OctComputation and LanguageComputer Vision and Pattern Recognition
The gist
Modern AI models that understand documents mix images, text, and layout information but are hard to explain, especially for important uses. The authors created a framework called DocXEval that tests many explainability methods specifically for these multimodal document models. They found that only a few methods work well consistently, with one called Occlusion standing out. Their work also shows how much each part of the document helps the model and points out dataset quirks, providing a useful guide for improving explainability in these systems.
Open → 2610.36929v1

Hypergraph method improves evidence selection for long multimodal documents

Hypergraph-Enhanced Evidence Selection for Long-Context Multimodal Document QA

Abstract: Long-context multimodal document question answering (QA) has become an important yet challenging task for modern vision-language models (VLMs), as it requires reasoning over evidence dispersed across pages and modalities within limited input budgets. Current pipelines adopt retrieval-augmented generation (RAG) to select multimodal evidence for downstream VLMs. However, existing systems mainly rely on static selection rules over fixed-granularity retrieval units such as pages or blocks, resulting in information loss during retrieval and suboptimal performance in downstream reasoning. To alleviate these issues, we formulate the core task as budget-aware evidence selection over structured document evidence and propose HyperDoc, a document RAG framework that constructs compact yet sufficient evidence while preserving document relations. Specifically, HyperDoc constructs a document hypergraph to explicitly capture structural, logical, and cross-modal relationships among multimodal document elements. Over this structured evidence space, HyperDoc performs adaptive cost-aware hyperedge selection based on query intent and input budget. Extensive experiments on three representative benchmarks demonstrate that HyperDoc outperforms various types of state-of-the-art baselines.

Mon 26 OctInformation RetrievalDatabases
The gist
Answering questions from long documents that contain both text and images is hard because important clues are scattered and systems have limited space to process them. The authors propose a new method called HyperDoc that organizes document parts into a network showing their relationships and picks the most helpful parts based on the question and how much information can fit. This approach helps keep important connections and reduces information loss. Tests show that HyperDoc works better than previous methods on several datasets.
Open → 2610.35542v1

Visual evidence use boosts multimodal document question answering accuracy

Loud and Clear: Making Visual Evidence Count in Multimodal Document RAG

Abstract: Multimodal document retrieval-augmented generation (RAG) has focused largely on retrieval, yet its answers depend on how well a vision-language model reads the retrieved pages. We find that evidence in retrieved pages remains underused even when the correct pages are retrieved. On SciEGQA oracle pages, five prior methods give the generator 0.7K to 2.7K visual tokens per query on average under a common image processor, yet raising the page-only budget from 1K to 32K improves accuracy by 15.22 points. At a fixed 16K budget, adding ground-truth crops improves accuracy by 3.75 points over page-only input. We study what the generator needs from retrieved pages through two requirements: Loud, making full use of visual token capacity, and Clear, refining the page so that its evidence becomes prominent. For Loud, increasing either the visual token budget or visual fidelity, the image resolution those tokens preserve, improves accuracy with the other held fixed, and full-page scaling outperforms repetition and tiling at matched budgets. For Clear, suppressing surrounding content improves accuracy without enlarging selected regions, and adding OCR of these regions to page images outperforms either input alone. We implement both as Loud and Clear (LoC), which combines higher-resolution pages with OCR from query-relevant regions without ground-truth annotations or additional training. LoC outperforms eight prior methods on SciEGQA, ViDoRe-v3, and MMLongBench-Doc, improving over the strongest, ColQwen3, by 8.88, 3.09, and 4.22 points with the same retriever and generator. It also outperforms its page-only variant under the same visual token budget. These results locate a bottleneck beyond retrieval: how the retrieved pages are given to the generator.

Thu 15 OctComputer Vision and Pattern RecognitionArtificial Intelligence
The gist
When computers answer questions using documents with both text and images, they often find the right pages but don't fully use the pictures to find the answers. The authors show that giving the computer clearer and more detailed images from these pages helps a lot. They introduce a way to make important parts of images stand out and add text reading from those parts, which makes answers more accurate. Their approach works better than earlier methods on several document question answering tests.
Open → 2610.21318v1

Visual evidence grounding improves question answering in long documents

VELD-QA: A Large-Scale Dataset for Visual Evidence Grounding and Question Answering in Long Documents

Abstract: Existing long-document QA benchmarks rarely combine large-scale human-authored questions with detailed visual evidence annotations, limiting diagnosis of evidence localization and interpretation failures. We present VELD-QA, a large-scale benchmark for Visual Evidence grounding and question answering in Long Documents. It contains 15,000 fully human-authored question-answer pairs across 5,859 real-world PDFs totaling 372,445 pages, balancing chart, table, and complex-layout tasks. Human annotations identify supporting pages and evidence elements and describe cited visual content, enabling separate retrieval and interpretation evaluation. We evaluate nine vision-language models under four evidence settings and seven retrieval-augmented and agentic systems. Our analysis identifies bottlenecks in evidence localization and interpretation. Retrieval-based methods substantially underperform the oracle setting with ground-truth supporting pages; full-document access does not close the gap. Localization alone is insufficient: oracle accuracy peaks at 81.7%, highlighting persistent visual interpretation and reasoning challenges. Enriching inputs with our visual-element annotations narrows the performance gap by 17-24%, demonstrating value beyond evaluation and motivating data enrichment for multimodal document QA. Our code is available at https://github.com/oracle-samples/veld-qa.

Wed 14 OctComputer Vision and Pattern RecognitionComputation and Language
The gist
Answering questions about long documents with lots of pictures, charts, and tables is hard because models need to find the right visual evidence first, then understand it. The authors created a big dataset called VELD-QA with 15,000 questions about real PDFs that have detailed markings showing which pages and visuals help answer each question. They tested various models and found that even when models know the right pages, understanding visual details and reasoning is still a big challenge. Adding detailed visual clues helps models get closer to the best possible performance, showing that such data is useful for improving these systems.
Open → 2610.20964v1

Multimodal graph reasoning improves complex document question answering

TopoGraphRAG-Bench: Evaluating Multimodal GraphRAG on Layout-Grounded Evidence Reasoning

Abstract: Real-world documents distribute evidence across text, tables, figures, and captions within complex page layouts. Answering complex questions over such documents therefore requires more than retrieving relevant passages: systems must recover the evidence topology that connects heterogeneous evidence units. Existing GraphRAG evaluations remain largely text-centered, while multimodal document RAG benchmarks assess cross-modal retrieval and generation without directly evaluating recovery of the intended evidence topology. We introduce TOPOGRAPHRAG-BENCH, a layout-grounded benchmark for multimodal evidence reasoning in GraphRAG, comprising 2,024 questions over 201 long, visually rich documents. Questions are constructed bottom-up from text, figure, and table evidence units under three controlled topologies: single-hop retrieval, bridge-chain reasoning, and multi-source synthesis. To ensure that questions preserve their intended structure, we apply counterfactual validation for shortcut resistance, modality necessity, and evidence necessity. We evaluate text-only GraphRAG, page-level visual retrieval, and multimodal GraphRAG systems using retrieval, generation, and topology-aware reasoning metrics. Multimodal GraphRAG systems achieve the strongest overall performance, but still fail when visual-textual evidence alignment or multi-unit composition is incomplete. Text-only GraphRAG struggles when key dependencies are grounded in figures or tables, while page-level visual retrieval lacks the fine-grained structure needed for topology recovery. These findings motivate GraphRAG systems that move beyond text-derived entity relation graphs to explicitly model document layouts, cross-modal evidence alignment, and the reasoning roles of evidence units. Code and data are available at https://richardlrc.github.io/TopoGraphRAG-Bench/.

Wed 7 OctArtificial IntelligenceComputation and Language
The gist
Answering complicated questions from documents that have lots of text, tables, pictures, and captions is hard because the information is spread across different parts and types. The authors created a new test called TOPOGRAPHRAG-BENCH to check how well computer systems can connect these different pieces of information visually and textually to find the right answers. They tested several systems and found that those using both images and text work best but still struggle to fully understand how different parts relate. Systems that only use text or only look at whole pages don't do as well. This shows that future methods need to better understand how document layouts and different types of information fit together.
Open → 2610.09360v1

Compact model with program harness matches large models for document math

When Harness Beats Scale, and When Reading Beats Both

Abstract: We describe our system for DocSem, the document-grounded quantitative reasoning shared task at DocInsights 2026, and analyze why it succeeded on labeled data and failed on the test set. The pipeline pairs hybrid block retrieval with Program-of-Thoughts (PoT) generation executed in a sandboxed interpreter, self-consistency sampling, and entity enrichment from chunk-level knowledge graphs. On our held-out split, application architecture moved the metrics far more than model scale did: PoT added 0.282 joint accuracy to a compact 7B model but at most 0.005 to a 72B model, and a 27B model with the full harness matched the 72B (0.884 vs.\ 0.873) at roughly 2.7$\times$ fewer parameters and a quarter of the CO$_2$. We read this through a distinction between world knowledge, which scales steeply with parameters, and language knowledge, which scales gently, and show that structured-output training makes a compact model harness-ready rather than merely small. On the raster, watermarked test PDFs the same system collapsed to 13.58\% joint (rank 149 of 163); a controlled re-rendering of the validation set reproduces the OCR half of the collapse while bounding what the simulation misses. Auditing the physical nature of evaluation inputs precedes architecture, and the leaderboard's bimodality is consistent with reading quality, not reasoning, having separated the field.

Mon 28 SeptComputation and LanguageArtificial IntelligenceInformation Retrieval
The gist
Solving math problems based on documents usually relies on very large AI models, which are costly and slow. The authors show that using a clever setup—called a program-of-thoughts harness—helps smaller models perform just as well as much bigger ones on labeled data. They also found that problems with reading documents from low-quality PDFs caused many systems to fail in tests, showing that how well a system reads input matters more than its reasoning ability. This means improving reading quality can be more important than making models bigger.
Open → 2609.34366v1

Page images improve document QA accuracy but increase latency and cost

Text, Pixels, or Both? Evaluating Input Representations for Multimodal Document QA

Abstract: Every document QA system begins with a choice that is rarely studied on its own: whether to feed the model page images, extracted text, or both. We isolate this choice, holding the prompt, judge, and scoring pipeline fixed, across four commercial model endpoints, two corpora, and two context regimes (gold evidence pages and the full document). On documents that fit the image budget, page images lead on accuracy at every document length on both corpora, but this advantage carries a growing latency and cost premium: text latency stays roughly flat as documents lengthen while image latency rises steadily. Text and images also fail on different questions, with exactly one representation correct on 19--25% of items across the reported cells, so neither subsumes the other. Exploiting this complementarity, a lightweight TF-IDF router that reads only the question text gains 2.6 points over always-text while cutting median latency 30% relative to always-vision, on a document-disjoint held-out split.

Fri 18 SeptArtificial Intelligence
The gist
When computers answer questions about documents, they can use either the words extracted from the pages, images of the pages themselves, or both. This paper shows that using page images generally helps computers answer more accurately but takes more time and computing power as documents get longer. Interestingly, some questions are answered correctly only by looking at images or only by looking at text, so using both can be better. The authors also built a simple system that chooses between text or image input based on the question, improving speed and accuracy.
Open → 2609.22628v1