Large language models improve recommendation accuracy with confident direct choices
Decision-Native Recommendation with Large Language Models
Information Retrieval
Summary
Making good recommendations means picking the right next thing a user would want, but some methods only suggest items indirectly or get confused by how choices are arranged. The authors propose a way to directly pick the best item from a list using large language models, making sure the choice does not change if the list order changes. Their method also provides a confidence level that shows how sure the system is about the recommendation. This approach is faster and more accurate than older methods, helping systems know when a recommendation is reliable or when it might need extra checking.
What this means in practice
- •For e-commerce platform engineers: Provide faster and more accurate personalized product recommendations with confidence scores to improve user satisfaction.
- •For mobile app developers: Integrate efficient, order-consistent item selection models that speed up recommendation generation within app environments.
Authors
Daoxuan Liu, Jiayi Liao, Jiancan Wu, An Zhang, Xiang Wang, Xiangnan He
Abstract
Large language models (LLMs) bring world knowledge to user preference modeling in sequential recommendation, but generation-based recommenders express next-item predictions indirectly as text. Generated outputs may fail to identify a unique candidate item, depend on candidate presentation order in the prompt, and lack explicit confidence in the recommendation. Recent studies explore Jev, a model for fast option selection with explicit confidence. Inspired by Jev, we introduce decision-native recommendation, a new paradigm that jointly models item choice and recommendation confidence using LLM representations. It requires valid candidate selection, consistency under candidate reordering, and confidence that reflects recommendation accuracy. We propose Direct Item Choice Estimation (DICE), which learns a personalized next-item distribution over supplied candidates from user interaction history and recommends the highest-scoring item with calibrated confidence. Tree-structured attention shares history computation across isolated candidate branches for order-consistent candidate scoring in a single forward pass. On three recommendation datasets, DICE achieves higher top-one accuracy than all evaluated baselines. Separate evaluations show unchanged item choices under tested candidate reorderings and end-to-end speedups of up to approximately $6\times$ over the evaluated generation-based methods. Its confidence estimates support selective recommendation by identifying more reliable predictions and guide additional computation for low-confidence examples in offline routing.