Efficient fine-tuning of language models speeds up recommendation systems
Merge and Branch: Efficient Tree-Structured Supervised Fine-Tuning of LLMs for Recommendation
Information RetrievalArtificial Intelligence
Summary
Recommendation systems sometimes use large language models (LLMs) to read and understand a user’s long history of actions, but this can be slow because the model processes a lot of text every time. The authors designed a new method called Merge and Branch, which compresses this history into a small set of tokens and then predicts multiple recommendations from this compressed form. This approach organizes predictions in a tree structure, allowing the model to efficiently share computations and train faster. Their method improves both the accuracy and speed of recommendation systems.
What this means in practice
- •For recommendation system engineers: Speed up training and improve accuracy of recommendation models by compressing user history and predicting multiple targets efficiently.
- •For interactive language model developers: Implement tree-structured caching to reduce computation cost during constrained decoding in language-based recommendation or dialogue tasks.
Authors
Yunqin Zhu, Yao Xie
Abstract
LLM-based generative recommendation reads long, text-rich interaction histories but generates only short semantic identifiers (SIDs). This asymmetry makes supervised fine-tuning inefficient: encoding the history is expensive, yet each history usually provides supervision for only one target. We present Merge and Branch (M&B) to reduce input cost and learn more from each encoded history. Merge compresses each historical item's SID and text into a small number of learned tokens. Branch uses the compressed history to predict multiple target SIDs. To share history computation while keeping these targets independent, M&B organizes each example as a shallow, wide tree. We develop tree FlashAttention to execute this layout efficiently, processing the shared history once and propagating gradients from every branch for end-to-end training. A tree-structured key/value cache extends history sharing to constrained decoding. Experiments on public benchmarks show that M&B outperforms strong recommendation baselines while substantially improving training and decoding efficiency.