Sequential recommendation improved with cross-user behavior and token-level credit
RPSD: Retrieval-Privileged On-Policy Self-Distillation with Optimal Transport
Artificial Intelligence
Summary
Sequential recommendation tries to predict what a user will want next based on their past choices. The authors point out two problems with current methods: they give the same credit to all parts of a sequence of suggestions and only learn from one user's data at a time. They propose a new technique called RPSD that uses a teacher model to give detailed feedback on each recommendation step and includes information from similar users’ histories. To do this, they measure similarities between user behaviors over time with a method called optimal transport. Their approach improves the accuracy of recommendations on real data.
What this means in practice
- •For e-commerce platform developers: Enhance product recommendation systems by using detailed feedback and cross-user behavior to boost accuracy and personalization.
- •For music streaming service engineers: Improve next-song predictions by incorporating information from similar listeners’ behavior while assigning more precise credit to the generated recommendations.
Authors
Dongyue Li, Zebang Cheng, Zhongyi Deng, Hongboxu, Yi Bin, Fei Ma
Abstract
Sequential recommendation predicts a user's next interaction from historical behavior. Recent LLM-based methods recast this task as generative recommendation and increasingly adopt reinforcement learning such as GRPO for post-training. However, these methods suffer from two limitations, namely credit-blind optimization, where a sequence-level reward is shared across all generated tokens, and collaborative-blind supervision, where each training signal is derived from a single user and ignores behavioral patterns shared across similar users. To address both issues, we propose Retrieval-Privileged on-policy Self-Distillation (RPSD). For credit-blind optimization, we introduce on-policy self-distillation (OPSD), in which a teacher with privileged knowledge re-evaluates the student's on-policy response, and the token-level confidence difference between the student and teacher reshapes the shared sequence-level advantage into token-specific credit. However, the privileged knowledge in vanilla OPSD is still derived from a single user, leaving collaborative-blind supervision unresolved. We therefore ground the teacher in collaborative behavioral evidence from similar users. Specifically, we use temporally aware optimal transport (OT) to compare interaction trajectories across users by jointly considering item similarity and temporal structure, and further apply diversity-aware selection to retain complementary trajectories as privileged evidence. In this way, on-policy self-distillation provides fine-grained token-level credit, while OT enriches the teacher with structured cross-user evidence. To the best of our knowledge, this is the first work to bring on-policy self-distillation to sequential recommendation and to use cross-user behavior as its privileged signal. Extensive experiments across multiple real-world datasets demonstrate that RPSD achieves state-of-the-art performance.