Large recommendation models made efficient through shared sequence encoding

SepLRM: A Foundation Model Paradigm for Recommendation

Information Retrieval

Summary

Recommendation systems try to predict what you might like based on your past behaviors, but as they get bigger, they become slower and cost more to run. The authors propose a new way called SepLRM that splits the recommendation work into one big shared part and several smaller specialized parts. This method saves effort by reusing detailed user behavior information across different recommendation tasks without losing accuracy. Their experiments show this approach works well on large datasets and improves real-world recommendation services at ByteDance.

What this means in practice

  • For e-commerce platform engineers: Increase recommendation system efficiency across diverse product categories by reusing user behavior encodings shared among multiple recommendation models.
  • For video streaming service developers: Deploy efficient personalized content recommendation by separating sequence encoding and task-specific ranking to reduce computation costs without accuracy loss.

Authors

Di Chen, Xijun Xiao, Qin Ren, Weijiang Lai, Zhijian Pan, Zhentao Pan, Pengfei Nie, Cheng Yuan, Zixuan Tan, Xuyang Liu, Tao Wang, Yunwen Huang, Hao Fang, Yichao Wang, Hao Zhang, Siyuan Wu, Hao Chen, Han Diao, Mingyu Yang, Yongkang Feng, Beihong Jin, Jinfeng Yang, Deping Xie, Peng Xu, Zheng Chai, Yuchao Zheng

Abstract

Large recommendation models (LRMs) improve recommendation performance by scaling behavior sequence length, model capacity, and training compute. However, the return on investment (ROI) of continued scaling deteriorates as marginal gains diminish and long-sequence encoding costs rise, especially when this computation is repeatedly executed by distinct models for diverse recommendation stages and business scenarios. To address this issue, we propose SepLRM, a recommendation foundation model (RFM) paradigm that identifies reusable sequence encoding and reorganizes multiple end-to-end (E2E) LRMs into one RFM and multiple independently optimized vertical models (VMs). The RFM adopts an encoder-decoder architecture and exports its final-layer encoder state as the shared key-value (KV) cache that is independent of downstream task-offered candidates. Unlike candidate-specific logits or aggregated user embeddings, the KV cache preserves fine-grained behavioral information. Separation at this native KV interface also allows each VM to construct candidate-conditioned queries while retaining the same query-KV interaction as its E2E counterpart, enabling high-fidelity capability transfer. To ensure both transfer effectiveness and system efficiency, we co-design cache compression, the training and serving workflows, and LG-Store, a distributed storage system for KV caches. Experiments on three large-scale datasets show that SepLRM achieves state-of-the-art offline performance with a transfer rate of approximately 100%. It also demonstrates strong transferability across model iterations, recommendation stages, and business scenarios. Deployment in two large-scale industrial scenarios substantially improves the ROI of sequence scaling and delivers consistent gains in online A/B tests at ByteDance.