Large language models outperform multi-token methods in personalized recommendations

Personalized LLM Recommendation with Semantic IDs: Autoregressive Beam Search vs. Multi-Token Prediction

Information RetrievalArtificial IntelligenceComputation and LanguageMachine Learning

Summary

Personalizing recommendations using large language models (LLMs) involves generating short codes that represent items. The authors compared two ways to produce these codes: making predictions one token at a time using beam search, or predicting all tokens simultaneously with a multi-token method. They found that the single-token beam search method performed better on recommendation accuracy and answering content questions, though generating predictions simultaneously was faster. They also improved training speed and addressed some language prediction setbacks with extra learning steps.

What this means in practice

  • For streaming service engineers: Improve item recommendations and content question answering accuracy in large streaming media catalogs using LLMs with semantic ID generation.
  • For cloud infrastructure teams: Reduce training costs and speed up model development by adopting single-head autoregressive beam search over multi-token prediction for large-scale personalized recommenders.

Authors

Sejoon Oh, Fernando Amat Gil, Evan Smothers, Dan Iter, Mark Thornburg, Dawit Mureja Argaw, Sourabh Medapati, Thea Wang, Ria Verma, Qiuling Xu, Harshit Syal, Ehsan Gholami, Moumita Bhattacharya, Ashish Rastogi

Abstract

Large-scale personalization is shifting from embedding-table retrieval to LLM-based recommenders that generate the Semantic ID (SID) of the item to recommend, where each SID is a short sequence of discrete hierarchical content codes. A single instruction-tuned model can then unify retrieval, content understanding, and language, and recommend brand-new titles from their content alone using SID; then, how should the model generate the SID? Traditionally, models rely on a single language modeling head that generates tokens autoregressively, typically using constrained beam search. In contrast, an emerging body of work explores multi-token prediction (MTP), where dedicated heads predict all tokens in parallel, substantially reducing the decoding latency associated with beam search. We report an industry-scale, apples-to-apples comparison of these two generation paradigms: single-head autoregressive decoding vs. MTP. We introduce a new auxiliary ranking loss, defined over the joint SID score, to train the MTP objective. We evaluate the SID training objectives on both quality and training/inference cost, inside a multi-task LLM that jointly does personalized retrieval, content question-answering (QA), and instruction following over a large streaming-media catalog. Our experiments show single-head beam search consistently beats MTP on most product-critical metrics: ranking and content QA, and trains at roughly 2.6x lower dollar cost on Qwen3-8B scale; MTP wins only on raw inference throughput. Around this finding, we detail an efficient training and inference stack and report end-to-end gains over the previous non-SID baseline equipped with an auxiliary ranking head: +77% item cold-start ranking, +22% content-QA accuracy, +2.8x faster training, with language metric regression traced to SID vocabulary expansion. We address the language regression by reinforcement learning post training.