Large language models improve text search with faster sparse retrieval
Aligning LLMs with Sparse Retrieval via Adaptive Sparse Supervision
Information Retrieval
Summary
Text search systems need to quickly find the most relevant documents from large collections. The authors found that big language models can help improve these searches by focusing only on the most important words, which makes the search faster and the index smaller. Their method adjusts which words to keep based on the input, rather than using a fixed number. Tests show this new approach works better than older methods, needing much less storage and making searches much quicker.
What this means in practice
- •For search engine developers: Build faster and smaller text search indexes that keep important keywords using adaptive sparse retrieval from large language models.
- •For digital library operators: Improve user search experience in large document collections by reducing response time while maintaining search quality.
Authors
Tongna Chen, Minghan Li, Guodong Zhou
Abstract
While large language models (LLMs) offer strong semantic representations, their application to first-stage retrieval incurs substantial latency and storage costs. Neural sparse retrievers such as SPLADE provide efficient retrieval but rely on relatively compact encoders with limited representational capacity. To bridge this gap, we propose an LLM-based sparse retrieval framework with input-adaptive sparse alignment. Specifically, we introduce a dynamic $α$-truncation strategy that selects input-specific supervision supports based on the cumulative activation mass of SPLADE representations. This strategy guides the LLM to preserve salient lexical signals while suppressing less informative activations, without relying on a fixed support budget. Subsequent retrieval fine-tuning combines contrastive learning, knowledge distillation, and sparsity regularization to improve retrieval effectiveness. Experiments on MS MARCO and six selected BEIR datasets demonstrate improvements over SPLADE++ while maintaining competitive effectiveness against LLM-based sparse retrievers. Compared with Lion-sp, our method reduces index storage by 6.4-fold and end-to-end query latency by over 11-fold, achieving a favorable effectiveness-efficiency trade-off.