SmartGR: Hierarchy and Beam-Aware Knowledge Distillation for Generative Recommendation
2026-08-03 • Information Retrieval
Information Retrieval
AI summaryⓘ
The authors focus on making recommendation systems better and faster by teaching a smaller model to copy a larger one, a process called knowledge distillation. They identified two problems unique to generative recommendation models: differences in learning difficulty across categories and errors during prediction search. To solve this, they created SmartGR, which carefully teaches the smaller model about category relationships and ranking preferences. Their tests show SmartGR improves recommendation accuracy by 8.6% and speeds up predictions by more than twice.
Generative RecommendationKnowledge DistillationSemantic ID HierarchyBeam SearchBeam-Aware Ranking DistillationHierarchy-Aware DistillationRecommender SystemsModel CompressionInference SpeedupBenchmark Datasets
Authors
Ziheng Zhang, Yu Cui, Bohao Wang, Yong He, Chao Yu, Chuan Yuan, Wujie Sun, Can Wang, Jiawei Chen
Abstract
Generative recommendation (GR) has emerged as a promising paradigm for recommender systems. Scaling up GR models can improve recommendation performance, but it also substantially increases inference cost. Knowledge distillation provides a practical solution by transferring knowledge from a large GR model to a lightweight one. However, existing distillation methods do not account for two GR-specific challenges: imbalanced distillation difficulty across the semantic ID (SID) hierarchy and incorrect prefix pruning during beam search. To address these challenges, we propose SmartGR, a novel distillation framework that utilizes Hierarchy-Aware SID Distillation to transfer the teacher's modeling capability across the hierarchy and leverages Beam-Aware Ranking Distillation to distill the teacher's ranking preferences during beam search. Extensive experiments on four benchmark datasets demonstrate the effectiveness and efficiency of SmartGR, improving the performance by 8.6% while achieving a 2.39$\times$ inference speedup on average.