TAAL: Mitigating Early Beam Pruning in Generative Recommendation via Temporal Autoregressive Alignment
Information RetrievalArtificial Intelligence
Summary
The authors study a way to recommend items by predicting parts of item codes step-by-step, but found that most prediction mistakes happen very early. They propose a method called Temporal Autoregressive Alignment (TAAL) that helps the model better handle early decisions by learning from past transitions and adjusting scores during prediction. This approach improves recommendation accuracy and reduces early errors across multiple public datasets. Their results show that TAAL is especially helpful when the model has fewer guesses to choose from during prediction.
Authors
Lianjie Li, Zhiying Tu, Dianhui Chu, Hongliang Sun
Abstract
Generative recommendation encodes items as hierarchical semantic identifiers (SIDs) and retrieves the next item through autoregressive decoding. Standard next-token prediction, however, does not explicitly cover the multimodal transitions present in interaction sequences, leaving the ground-truth SID vulnerable to irreversible pruning at early beam-search branches. Across three public benchmarks, we find that 91.9\%--96.6\% of retrieval failures occur within the first two decoding steps. We therefore propose Temporal Autoregressive Alignment (TAAL). During training, TAAL constructs a joint $(c_1,c_2)$ soft target from historical transitions and aligns the early-prefix distribution with a forward KL objective. During inference, it calibrates candidate scores with pointwise mutual information (PMI) to reduce the influence of globally frequent prefixes. On Amazon Beauty, Instruments, and Yelp, TAAL improves NDCG@10 over the standard baseline by 39.5\%, 6.7\%, and 28.6\%, respectively, while increasing full-SID survival by 3.9\%--16.6\%. Beam-width analysis further shows that the relative survival gain grows as the beam narrows, reaching 39.4\% at $B=5$.