Fixed budget interaction speeds up multi-vector search with less storage
FLINT: Fixed-budget Late Interaction
Information RetrievalComputation and Language
Summary
Matching search queries and documents at the word level can find better results but needs lots of storage and is slow. The authors present FLINT, which compresses queries and documents into a fixed number of vectors using attention pooling. This reduces storage and speeds up search while keeping similar or better accuracy compared to other methods. FLINT works well especially with long, complex queries and makes searching large collections more efficient.
What this means in practice
- •For search engine developers: Create more efficient multi-vector search systems that handle long complex queries faster and with smaller storage.
- •For digital library managers: Reduce storage footprint and speed up document retrieval in large-scale collections supporting detailed token-level matching.
Authors
Himank Handa, Negar Arabzadeh, Matei Zaharia
Abstract
Multi-vector retrieval models such as ColBERT often outperform single-vector models of comparable scale by matching queries and documents at the token level. However, retaining a vector for every token substantially increases storage requirements and slows retrieval compared to single-vector bi-encoders, limiting their scalability. We introduce FLINT (Fixed-budget Late INTeraction), an architecture that uses attention-based pooling to represent each query and document with a fixed number of embedding vectors. Unlike architectures that pool only document representations while retaining token-level queries, FLINT applies pooling symmetrically to both sides, limiting document storage and the number of vector comparisons needed during retrieval. FLINT achieves comparable ranking effectiveness to the closest baselines on BEIR and outperforms them on several benchmarks with longer and more complex queries, including BrowseComp+, CRUMB, and TREC Tip-of-the-Tongue 2025. We demonstrate that FLINT's effectiveness-efficiency benefits extend across encoder architectures, testing it on two separate backbones (BERT and ModernBERT). On long-query corpora, FLINT delivers 1.5-2.8 times faster end-to-end retrieval than ConstBERT, the closest baseline. FLINT also reduces index size by 5.3-9.3 times relative to token-level late-interaction models such as ColBERTv2.