Late interaction search models simplified with spectral compression techniques
EigenLI: Spectral Approximations to Late Interaction
Information RetrievalMachine Learning
Summary
Some search systems represent documents with many small pieces to improve accuracy, but this takes a lot of storage and slow comparisons. The authors discovered that these detailed pieces actually lie mostly in simpler, smaller spaces. They created a method called EigenLI that finds these smaller spaces for each document to reduce size and speed up ranking without losing much accuracy. Their approach works better than common ways of grouping document pieces, and also creates compact single-vector versions that improve fast approximate search. This helps make powerful search methods more practical for large-scale use.
late-interaction modelsColBERTtoken embeddingslow-rank structurespectral approximationeigenvectorsclusteringMaxSim scoringapproximate nearest neighborsdocument retrieval
Authors
Archish S, Sabyasachi Basu, Ankit Garg, Ravishankar Krishnaswamy, Kirankumar Shiragur
Abstract
Late-interaction models such as ColBERT achieve strong effectiveness by representing each document with many token-level vectors, but this expressivity leads to large indexing cost, storage footprints and expensive MaxSim scoring. We show that late-interaction representations exhibit an intrinsic low-rank structure: document token embeddings concentrate in a low-dimensional subspace that preserves most of the retrieval signal. Leveraging this observation, we introduce EigenLI, a spectral approximation framework that compresses late-interaction representations via document-specific low-dimensional subspaces. Unlike clustering or pooling methods, EigenLI identifies the dominant eigendirections of each document and uses them to construct reduced interaction representations. Empirically, $k$-EigenLI with $k \le 32$ outperforms k-means and Ward clustering based pooling methods on ColBERTv2 and AnswerAI-ColBERT-small; GTE-ModernColBERT exhibits a different tradeoff at $k=32$, where clustering methods perform better. The same spectral construction also yields EigenLI-SV, an ANN-compatible single-vector representation derived from the second-order summary of the reduced structure. Across multiple datasets and all three text models, EigenLI-SV consistently outperforms comparable single-vector surrogates such as MUVERA.