Spectral method cuts memory and speeds scoring in text search
EigenLI: Spectral Approximations to Late Interaction
Information RetrievalMachine Learning
Summary
Searching documents by comparing many small pieces can be very accurate but also slow and use lots of memory. The authors found that these many pieces actually fit well into a smaller space without losing much important information. They created EigenLI, a way to shrink these pieces using mathematical directions called eigenvectors, making search faster and simpler. Their tests show this method beats other ways of shrinking data and even creates compact single summaries that work well for approximate search.
What this means in practice
- •For search engine engineers: Develop document retrieval systems with reduced memory and faster scoring by applying spectral approximations to token-level representations.
- •For natural language processing developers: Create compact single-vector representations for fast approximate nearest neighbor search in text retrieval tasks.
Authors
Archish S, Sabyasachi Basu, Ankit Garg, Ravishankar Krishnaswamy, Kirankumar Shiragur
Abstract
Late-interaction models such as ColBERT achieve strong effectiveness by representing each document with many token-level vectors, but this expressivity leads to large indexing cost, storage footprints and expensive MaxSim scoring. We show that late-interaction representations exhibit an intrinsic low-rank structure: document token embeddings concentrate in a low-dimensional subspace that preserves most of the retrieval signal. Leveraging this observation, we introduce EigenLI, a spectral approximation framework that compresses late-interaction representations via document-specific low-dimensional subspaces. Unlike clustering or pooling methods, EigenLI identifies the dominant eigendirections of each document and uses them to construct reduced interaction representations. Empirically, $k$-EigenLI with $k \le 32$ outperforms k-means and Ward clustering based pooling methods on ColBERTv2 and AnswerAI-ColBERT-small; GTE-ModernColBERT exhibits a different tradeoff at $k=32$, where clustering methods perform better. The same spectral construction also yields EigenLI-SV, an ANN-compatible single-vector representation derived from the second-order summary of the reduced structure. Across multiple datasets and all three text models, EigenLI-SV consistently outperforms comparable single-vector surrogates such as MUVERA.