Papers for

visual search developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Multimodal embeddings balance detail and efficiency with residual compression

ResComEmb: Effective and Efficient Multimodal Embedding via Residual Homogeneity Compression

Abstract: Multimodal large language models (MLLMs) have shown strong potential for universal multimodal representation learning. However, existing methods either compress each input into a single vector, limiting fine-grained expressiveness, or retain long sequences of visual-token vectors, incurring substantial storage and interaction costs. To resolve this trade-off, we propose ResComEmb, a trainable framework for effective and efficient universal multi-vector multimodal embedding. ResComEmb first encodes each input at native dynamic resolution into ordered global, intermediate, and fine-grained views. After MLLM contextualization and embedding projection, a trainable Residual Homogeneity Compression (RHC) module reduces within-granularity redundancy and cross-granularity repetition under explicit visual token budgets. Then, ResComEmb introduces a length-adaptive Bidirectional Late-Interaction Matching mechanism for robust query-document scoring, which averages the strongest token-level matches in each direction and combines the two scores using a weight based on how many valid tokens each side has. Extensive experiments on MMEB, ViDoRe V1, and ViDoRe V2 show that ResComEmb produces higher-quality universal multimodal embeddings than VLM2Vec-V2, and outperforms ColQwen2.5 in visual document retrieval using only 37.5% of its full visual token budget, demonstrating a favorable effectiveness-efficiency trade-off.

Tue 29 SeptComputer Vision and Pattern RecognitionArtificial Intelligence
The gist
Multimodal large language models struggle to represent images and text together efficiently—either they use one summary vector that misses details, or many detailed vectors that take a lot of space. The authors designed ResComEmb, a new method that compresses redundant parts of multimodal inputs while keeping important details by looking at different detail levels. This makes embeddings smaller and faster to use without losing quality. They tested it on image document tasks and found it outperforms previous methods while using fewer resources.
Open → 2609.37225v1