MAGIC method cuts storage needs in visual document search
MAGIC: Marginal-Guided Compression with Optimal Transport for Efficient Visual Document Retrieval
Computer Vision and Pattern RecognitionInformation Retrieval
Summary
Visual document retrieval means finding documents by looking at their images. The authors found that current methods use many small parts of the images but this takes up a lot of space and time. They created MAGIC, a way to shrink the data by focusing on the most important parts used during searching. This makes the search faster and uses less storage without needing to retrain the system. Tests showed MAGIC works better than older shrinking methods, especially when making the data much smaller.
What this means in practice
- •For digital archive maintainers: Reduce storage and speed up searches for large visual document collections by compressing page embeddings smartly without retraining.
- •For enterprise search engineers: Improve efficiency of document image search systems by applying MAGIC to balance search accuracy and system resource use under heavy data compression.
Authors
Xu Yuan, Hua Liu, Wenqi Fan, Qing Li
Abstract
Recent visual document retrieval (VDR) systems such as ColPali use multi-vector page embeddings, in which patch-level vectors enable fine-grained evidence matching but incur substantial index storage and MaxSim scoring overhead. Post-hoc merging offers a practical route to efficient VDR by reducing this cost without retraining the retriever, but its uniform reconstruction objectives are poorly aligned with the sparse, non-uniform patch usage induced by late-interaction retrieval. Under aggressive compression, this misalignment can preserve rarely used patches while concentrating retrieval activity on too few retained representatives. To address this misalignment, we propose Marginal-Guided Compression with Optimal Transport (MAGIC), a training-free post-hoc compressor for efficient retrieval with frozen multi-vector embeddings. MAGIC derives a MaxSim-induced compression surrogate and optimizes it through a two-marginal entropic optimal-transport formulation, where a retrieval-demand source marginal prioritizes high-use patches and a balanced target marginal regularizes retained-facet usage. Across ViDoRe benchmarks, keep ratios, and retrieval backbones, MAGIC consistently outperforms strong post-hoc compressors, with particularly large gains in the aggressive-compression regime; component ablations verify the complementary effects of its two marginals. We release the code at: https://github.com/xandery-geek/MAGIC.