Llm recommendation systems cut delay by thinking without words
Do Recommenders Need to Think in Words? Latent Reasoning for Efficient LLM-Based Recommendation
Computation and Language
Summary
Many recommendation systems use large language models (LLMs) that think aloud in text before giving suggestions, which slows them down. The authors propose a new way to train these models to think silently inside their 'minds' instead of writing out their reasoning, keeping the same accuracy but running much faster. Their method guides the model's internal steps during training so it learns to reason without producing extra words at prediction time. Tests on Amazon product recommendation tasks show their approach is both more accurate and much faster than previous methods relying on visible reasoning text.
What this means in practice
- •For ecommerce platform engineers: Build faster and more accurate product recommendation systems by training LLMs to reason internally, eliminating textual reasoning overhead during predictions.
- •For mobile app developers: Improve recommendation response times in apps by deploying LLMs trained with latent reasoning steps that reduce computational delay at inference.
Authors
Hongyang Su, Beibei Kong, Lei Cheng, Chengxiang Zhuo, Zang Li
Abstract
LLM-based recommender systems increasingly generate chain-of-thought (CoT) text before each item prediction to activate the LLM's reasoning capacity, costing tens to hundreds of extra tokens per recommendation at inference. We ask whether inference-time reasoning artifacts (textual or latent) can be replaced by training-time supervision alone, without loss of accuracy. We propose LatentRec, a framework for LLM-based generative recommendation that replaces inference-time reasoning artifacts with training-time per-step head supervision. LatentRec has three components. A 2.5M-parameter Continuous Thought Module iterates K latent reasoning steps over the LLM's hidden state during training. Soft Token Selection bridges the ~14x norm gap between hidden states and input embeddings. A Per-step Reasoning Loss (PRL) supervises every intermediate state through the LLM's prediction head against the first token of the target Semantic ID. The trained LLM retains this accuracy without the module at inference, so the module can be discarded at deployment. On three Amazon benchmarks, LatentRec improves HR@10 by 4.0-5.0% (relative) over a CoT-distilled baseline at 18.3x lower latency (batch=1). Per-step training supervision consistently improves accuracy across three datasets, paid once at training rather than at every recommendation.