Dense retrieval outperforms generative retrieval with static item values
Revisiting Generative vs. Dense Retrieval: A Static Value Prior Reverses the Gap
Information RetrievalArtificial IntelligenceMachine Learning
Summary
Finding the best products for people to buy online can be done using different computer methods. The authors compared two methods: one that generates recommendations using language models, and another that finds dense matching features. They discovered that the generation method seems better only because it focuses on popular items, but when adjusting for item values, the dense method actually works better and faster. They also showed that even very large language models don't beat smaller systems for finding relevant products.
What this means in practice
- •For ecommerce engineers: Use static item value scores in reranking dense retrieval candidates to improve product recommendation quality and recall.
- •For search system developers: Implement dense retrieval methods with static value adjustments instead of large generative models for efficient, accurate product search.
Authors
Renbo Tu, Yang Liu, Shuying Sun
Abstract
Generative retrieval (GR) has emerged as a strong candidate generator for sequential recommendation as language models and semantic-item identifiers have improved. However, GR is not a uniformly stronger replacement of dense retrieval (DR) in online retail candidate generation. We examine their differences on two Amazon Toys datasets under different metrics. GR leads on best relevant item value among users both methods cover, while DR leads on exact-target recall and relevant value across the ranked list. With deep analysis, we find that popularity contributes the most to GR's measured value lead. Adding the same static item value to DR in one reranking step reverses this lead and keeps recall above GR. Finally, on an industrial-scale internal e-commerce dataset with LLM-judged relevance, we show that a 14B SID-LLM still underperforms a $47\times$ smaller dense encoder.