Papers for
document ai engineers
Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.
Visprove improves detailed visual evidence attribution in scientific qa
VisProve: Learning Typed Visual Provenance for Scientific Document Question Answering
Abstract: Visual citations show where an answer is supported, but not how the cited content contributes to it. In scientific document question answering, a model may read a value directly, compress several observations, or combine evidence to infer a conclusion. We study typed visual provenance, where a system generates an answer together with source regions and explicit support relations. We introduce ReViE, a dataset of 3,941 questions over complete scientific documents, with page, section, and region annotations labeled as Quotation, Compression, or Inference. We further propose VisProve, which augments joint answer and provenance generation with differentiable Spatial Evidence Reading and Evidence-Answer Relation Feedback. VisProve conditions next-token prediction on localized visual evidence and its estimated support relation through a gated norm-bounded residual. On the document-disjoint ReViE test set, VisProve improves Typed-F1 from 3.52 to 5.48 and Section-F1 from 45.99 to 53.24 over joint supervised fine-tuning with the same backbone, while Page-F1 changes only marginally. These results suggest that coupling visual grounding with support relation modeling mainly improves fine-grained provenance, while the low absolute Typed-F1 reveals a substantial gap between answer generation and precise visual evidence attribution.
Adversarial watermarks can disrupt document AI without ruining readability
Visually Legible but Adversarial: Watermark-Constrained Attacks on End-to-End VLMs for Document Understanding
Abstract: Visible watermarks convey document ownership, confidentiality, and status, but can also carry adversarial signals. We introduce Auto-Regional Adversarial Watermarking (AR-AWM), a model-aware white-box attack on end-to-end visual language models (VLMs) for document understanding. AR-AWM constructs a small, spatially distributed watermark-text support and optimizes only its pixels to disrupt visual representations. The attack is untargeted and question-agnostic: generation uses no questions, answers, parsing prompts, language-model gradients, or language-model outputs. Each attacked page is reused across downstream queries. On Qwen2.5-VL-7B, a watermark support covering 7.98% of page pixels under the reporting threshold reduces DocVQA exact match from 0.899 to 0.150 and average normalized Levenshtein similarity (ANLS) from 0.939 to 0.212. Selected qualitative cases retain visually legible source fields. The same support without optimization and with matched random perturbations retains ANLS of 0.932 and 0.933, respectively, identifying adversarial optimization as the primary cause of degradation. Layout-matched optimized rectangles yield ANLS of 0.276, showing an incremental benefit from text-shaped geometry. AR-AWM also degrades all reported text, formula, reading-order, and table metrics on OmniDocBench. Separate white-box optimization on InternVL2.5-8B reduces ANLS from 0.922 to 0.046, while weak cross-model transfer reveals strong encoder specificity. These results identify conventional watermark-shaped supports as an adversarial surface for document VLMs in the evaluated white-box setting.luated white-box setting.