Adaptive sparsity cutting retrieval times and storage costs
Adaptive Sparsity Optimization with Learnable Soft Top-K and Per-Term Thresholding for Efficient Retrieval
Information Retrieval
Summary
Searching through large collections of documents can be slow and costly because each document and search query is represented by very large lists of important words. The authors show how to make these lists much shorter without losing search quality by using smart methods that learn which words really matter. This results in much faster searches and less space needed to store information, while still finding good matches. They tested their ideas on popular search datasets and improved on previous methods.
What this means in practice
- •For search engine engineers: Cut search query and document representations to speed up retrieval and reduce storage costs in semantic search systems using LLM expansions.
- •For enterprise it infrastructure teams: Shrink data footprint for large-scale document retrieval systems to lower hardware requirements and improve response times.
Authors
Wentai Xie, Parker Carlson, Shanxiu He, Tao Yang
Abstract
Recent work on neural sparse retrieval has demonstrated strong relevance by leveraging Large Language Models (LLMs) for semantic term expansion. However, learned models paired with previous sparsification techniques still yield overly long document and query vectors partly due to a large LLM vocabulary, imposing a serious challenge to retrieval time and space efficiency. This paper proposes a scheme for optimizing model sparsity through a synergy of adaptive strategies, including learnable soft top-K, per-term thresholding, and FLOPs regularization to increase the sparsity of query and document vectors. Experimental results with Lion-SP model on the MS MARCO and BEIR datasets demonstrate that the proposed scheme can outperform the baselines by significantly reducing the average query and document lengths. Our scheme can achieve much shorter retrieval latency and lower storage cost while maintaining highly competitive relevance.