Visual document retrieval improves accuracy and reduces storage needs
EVIE: Evidence-Vector-Informed Embeddings for Visual Document Retrieval
Information Retrieval
Summary
Finding specific information in complicated documents like manuals or reports can be slow and lose important visual clues. The researchers introduce EVIE, a new method that keeps important page details and organizes information more efficiently during search. EVIE combines smarter training, clever compression, and better filtering to find answers faster and use less storage. Their tests show EVIE works better than previous methods while using much less space.
What this means in practice
- •For enterprise search teams: Improve document search accuracy and speed in large-scale collections by using EVIE's efficient embeddings and compressed indexing.
- •For digital archiving specialists: Store large volumes of digitized documents with reduced storage costs by applying EVIE's hierarchical index compression.
Authors
Zifei Wang, Wei Wen, Qiang Ji, Qian-Wen Zhang, Ruizhi Qiao, Xing Sun
Abstract
Accurate and scalable visual document retrieval (VDR) requires both fine-grained page understanding and efficient indexing, yet existing approaches struggle to achieve both. OCR-based text retrieval adds preprocessing latency and can lose visual and structural cues needed to understand complex pages. Single-vector vision-language models bypass OCR, but compressing an entire page into one vector limits the granularity of query--document matching. Multi-vector retrievers with MaxSim provide finer interactions, yet demand large indexes and still leave room for accuracy improvements. We argue that overcoming these limitations requires preserving query-relevant page evidence throughout representation learning and index construction. To this end, we introduce \textbf{\textit{EVIE}} (Evidence-Vector-Informed Embeddings), a family of native visual document retrievers integrating three key innovations: (1) Evidence-judged data governance, which uses a multimodal judge to identify answer-bearing positives and filter unreliable negatives. (2) Bidirectional teacher--student learning with symmetric listwise distillation and prefix-based Matryoshka representation learning (Prefix-MRL), enabling one student checkpoint to serve six nested embedding dimensions without re-encoding. (3) Hierarchical agglomerative index compression (HAC), which clusters page tokens with spatial regularization and stores semantic centroids for single-stage MaxSim retrieval. Extensive experiments across 138 tasks from ViDoRe V1, V2, V3, and JinaVDR validate EVIE. EVIE-8B achieves 66.75 nDCG@10 on V3, exceeding the best external baseline by 1.43 points, with a four-suite average of 79.51. EVIE-4.5B with HAC retains 59.58 nDCG@10 at only 3.81 GiB per million pages, reducing vector payload by $128\times$. Together, these results improve the accuracy--storage trade-off for visual document retrieval.