Language models boost personalized recommendation speed and quality
DuoLM: Fast and Effective Personalized Recommendation with Language Models
Information Retrieval
Summary
Personalized recommendation systems help suggest items you might like based on your past behavior and item details. The authors present DuoLM, a method that speeds up these recommendations without losing accuracy by splitting tasks between two language models—one creating compact item summaries and another ranking items using user history. This approach reduces the online computation needed, making recommendations faster and able to handle many candidates at once. Their tests show that DuoLM matches expert systems in quality while being about 60 times faster on specialized hardware.
What this means in practice
- •For e-commerce platform engineers: Deploy a high-throughput recommendation system that processes millions of candidates quickly without sacrificing accuracy, improving user experience on retail websites.
- •For streaming service developers: Integrate fast personalized content ranking by encoding rich item data offline and combining it with user feedback for real-time video or music recommendations.
Authors
Jiang Wu, Yutong Song, Hongbo Wang, Guanghe Cao, Jeff Liang
Abstract
Language models can make personalized recommendations from rich item content, user history, and feedback. However, evaluating many candidates with full-text inputs incurs substantial online computation and limits serving throughput. We introduce DuoLM, a framework for fast and effective personalized recommendation that integrates a compact recommendation model with an efficient serving runtime. An encoder LLM learns compact item representations for offline reuse, while a personalized ranker LLM combines them with user history and feedback for recommendation. We train both models end to end with recommendation supervision and teacher distillation. The runtime further accelerates inference through virtual token IDs for embedding access, shared history computation, and CUDA Graph replay. Benchmark results show that our framework achieves recommendation quality comparable to that of the expert teacher model. On a single H100, it delivers approximately 26,400 candidate scores per second, about $60\times$ the throughput of the teacher served with vLLM and automatic prefix caching.