SA-RSQ: A Versatile Sparse Representation Framework for Multi-modal Recommender Systems
2026-08-24 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors address the problem of using large, complex data in recommendation systems, which usually takes up a lot of storage and slows things down. They introduce a new method called SA-RSQ that stores information more efficiently by carefully selecting important parts and assigning probabilities, allowing better learning and reconstruction without common shortcuts. Their tests on food-delivery ads show that this method balances storage size and performance well. Early experiments and a real-world test also indicate it can improve click rates and revenue.
Recommender SystemsQuantizationSparse ActivationTop-K RoutingSoftmaxCodebookClick-Through Rate (CTR)Storage BudgetA/B TestingNext-Distribution Prediction
Authors
Xiang Wang, Shigang Quan, Tingzhen Chang, Kang Yang, Sitong Chen, Yabo Fan, Xingxing Wang, Zhaodian He
Abstract
Deploying high-dimensional multimodal features in industrial recommender systems incurs substantial storage and latency overhead. Hard quantization is compact but introduces boundary distortion, whereas dense soft quantization couples representation quality to the limited storage budget. We propose Sparse Activation-based Residual Soft Quantization (SA-RSQ), which uses Top-K sparse routing and softmax weights to store compact (Index, Probability) tuples. The stored tuples decouple per-item storage from codebook dimensionality; for a fixed selected support, gradients propagate through the routing weights and weighted reconstruction without relying on a straight-through estimator. Experiments on a proprietary food-delivery advertising dataset show favorable reconstruction-performance and CTR trade-offs across storage budgets of 8-48 bytes per item. A preliminary Next-Distribution Prediction study and a one-week online A/B test further demonstrate the practical potential of SA-RSQ, with relative lifts of +2.51% in CTR and +3.66% in CPM.