Scaling and Stabilizing Large-Scale Embedding-Based Retrieval
2026-07-11 • Information Retrieval
Information Retrieval
AI summaryⓘ
The authors address problems in large-scale e-commerce search where models struggle to pick the best items from millions during real use, even if trained on smaller sets. They improved training by mixing different ways to find challenging negative examples and combined predictions with item info to catch subtle mistakes. They also carefully upgraded the model to a stronger one without losing useful knowledge from the old model. Their approach, tested both offline and live at Walmart, led to better search ranking and increased revenue.
embedding-based retrievaldual-encoderhard negative miningcross-encoderdistillationDistilBERTGTE-baseNDCG@5A/B testinge-commerce search
Authors
Zhen Yang, Juexin Lin, Hongwei Shang, Kaihao Li, Feng Liu, Satya Chembolu, Xunfan Cai, Xinyi Liu, Cun Mu, Tony Lee, Ciya Liao
Abstract
Embedding-based retrieval (EBR) is foundational to large-scale e-commerce search, yet its effectiveness is often constrained by the quality of training signals and the representational capacity of the encoder. Standard dual-encoders suffer from a training-inference gap: they are optimized on narrow candidate pools but must discriminate against hundreds of millions of items during inference. Furthermore, while transitioning to higher-capacity backbones can mitigate this gap, simply replacing a mature model can lead to inconsistent retrieval behavior and a loss of the domain-specific knowledge established in previous iterations. In this paper, we present a unified pipeline deployed at Walmart that addresses both signal quality and model evolution. Our contributions are two-fold: (1) Hybrid Hard Negative Mining: We integrate Online Cross-Batch Sampling to increase negative diversity by an order of magnitude and Hybrid Offline Mining, which combines cross-encoder predictions with metadata heuristics to identify nuanced mismatches. (2) Legacy-Aware Distillation: We transition from DistilBERT to a higher-capacity GTE-base encoder. To ensure a smooth and superior transition, we introduce a Warm-Start Distillation technique that transfers domain-specific expertise from the legacy model to the new backbone. Validated through extensive offline experiments and online A/B testing, the proposed pipeline is deployed in live production, delivering a +7.34% improvement in NDCG@5 and a +0.50% lift in gross revenue.