Semantic id method improves recommendation accuracy by combining content and preferences
SIDRec: End-to-End Semantic ID Representation and Modeling for Recommendation
Information RetrievalArtificial Intelligence
Summary
Making recommendations that fit your tastes can be tricky because systems need to understand both item details and what people like. The authors found that previous methods didn’t fully match item features with user preferences and training goals. They developed SIDRec, which learns a compact way to represent items by mixing what the items are about with what users prefer, all while training the recommendation system to optimize for better suggestions. Tests showed SIDRec made more accurate recommendations on several public datasets compared to other methods.
What this means in practice
- •For e-commerce developers: create recommendation engines that better match product details with user preferences, improving user engagement and sales.$Commercial implications: enables a product recommendation feature that outperforms previous systems by learning discrete item representations aligned with preferences
- •For video streaming engineers: enhance content recommendation by integrating item semantic features with viewer data for more relevant video suggestions.
Authors
Zhuohang Jiang, Yongsen Pan, Pangjing Wu, Zihuai Zhao, Zheng Hu, Yuxin Chen
Abstract
Semantic IDs (SIDs) have emerged as a promising approach for discriminative recommendation by encoding continuous item rep- resentations into compact discrete identifiers. However, existing SID-based methods face two fundamental limitations: semantic mis- match between content semantics and collaborative preferences, and objective mismatch between SID construction and downstream recommendation optimization. To address these challenges, we propose SIDRec, an end-to-end recommendation-aware SID learn- ing framework that jointly models content and collaborative in- formation while aligning SID construction with recommendation objectives. Specifically, SIDRec introduces a self-updating fusion mechanism that explicitly integrates pretrained content represen- tations with collaborative embeddings initialized from a trained recommendation model, enabling preference-aware semantic mod- eling. Furthermore, SIDRec employs an MMoE-inspired joint opti- mization mechanism with shared experts and task-specific gates, allowing recommendation supervision to directly guide differen- tiable quantization and produce task-adaptive discrete representa- tions. Extensive experiments on three public benchmark datasets demonstrate that SIDRec consistently outperforms all competing methods across both AUC and GAUC, achieving up to 5.15% relative improvements over the strongest baselines. Further ablation stud- ies validate the effectiveness and complementarity of collaborative semantic modeling and end-to-end SID optimization.