Recursive recommenders improve by selecting most useful verified sequences
From Valid to Useful: Post-Verification Acquisition for Recursive Self-Improving Recommendation
Information Retrieval
Summary
Recommender systems try to get better by generating new user activity sequences and retraining themselves with that data. However, not all generated sequences help learning equally well. The authors show that by carefully choosing which verified sequences to use based on where the recommender models disagree the most, they can improve recommendation quality more efficiently. Their method, DA-RSIR, achieves better results than simply using all verified sequences across various datasets and models. This approach separates verifying data from deciding which data truly helps improvements.
What this means in practice
- •For recommendation system engineers: Improve recursive retraining by selecting the most informative verified sequences to better personalize recommendations.
- •For machine learning platform teams: Integrate disagreement-based sample selection into model update pipelines to enhance data efficiency without extra labels or teacher models.
Authors
Tonmoy Hasan, Taylor Foust, Shao Tang, Leonardo Neves, Aman Gupta, Hiroto Udagawa, Helder Dias, Daniel Silva, Rohan Ramanath
Abstract
Sequential recommenders can generate synthetic interaction sequences and retrain on the augmented corpus in a recursive self-improvement loop. To limit error accumulation, current methods verify each generated sequence remains predictive of the user's real interactions and discard those that drift away from it. Verification does not, however, determine which verified sequences should train the next model. With every verified sequence used for training, source sequences yielding more verified sequences or longer continuations have more influence, although neither quantity indicates how much those sequences will help the next model. We formulate the decision of which verified sequences are used to train the next model as \emph{post-verification acquisition} and introduce {\bf Disagreement-Aware Recursive Self-Improving Recommendation (DA-RSIR)}. DA-RSIR caps each source sequence's contribution and ranks its verified sequences by how much the model's predictions disagree over their augmented interactions. It uses a score derived from Bayesian Active Learning by Disagreement (BALD) and estimated with Monte Carlo (MC) dropout. DA-RSIR requires no extra labels, teacher model, or quality scorer. Across four datasets, three recommender models, and two metrics, it improves on the retain-all approach in all $24$ comparisons and attains the highest mean in $23$ of $24$ overall; the aggregate improvement is statistically significant on both metrics. A single DA-RSIR round exceeds the retain-all approach's best gain over five recursive rounds. These findings establish post-verification acquisition as a separate control point in recursive self-improvement, separating which sequences pass verification from which verified sequences are used to train the next model.