IAT: Instance-As-Token Compression for Historical User Sequence Modeling in Industrial Recommender Systems
2026-04-10 • Information Retrieval
Information Retrieval
AI summaryⓘ
The authors introduce a new two-step method called Instance-As-Token (IAT) to improve recommendation systems by better summarizing users' past behaviors into compact tokens. First, they compress detailed features of each user interaction into a single embedding, using two ways to maintain order. Then, they use these tokens in sequence models to capture long-term user preferences. Their experiments show IAT works better than existing methods and transfers well across different recommendation tasks. This approach has also been successfully applied in real-world commercial systems like online shopping and live-streaming platforms.
sequence modelingrecommender systemsembeddinguser interactiontemporal orderuser-order compressionlong-range preferencescross-domain transferabilityinstance embeddingsequence features
Authors
Xinchun Li, Ning Zhang, Qianqian Yang, Fei Teng, Wenlin Zhao, Huizhi Yang, Heng Shi, Linlan Chen, Yixin Wu, Zhen Wang, Daiye Hou, Fei Qin, Lele Yu, Yaocheng Tan
Abstract
Although sophisticated sequence modeling paradigms have achieved remarkable success in recommender systems, the information capacity of hand-crafted sequential features constrains the performance upper bound. To better enhance user experience by encoding historical interaction patterns, this paper presents a novel two-stage sequence modeling framework termed Instance-As-Token (IAT). The first stage of IAT compresses all features of each historical interaction instance into a unified instance embedding, which encodes the interaction characteristics in a compact yet informative token. Both temporal-order and user-order compression schemes are proposed, with the latter better aligning with the demands of downstream sequence modeling. The second stage involves the downstream task fetching fixed-length compressed instance tokens via timestamps and adopting standard sequence modeling approaches to learn long-range preferences patterns. Extensive experiments demonstrate that IAT significantly outperforms state-of-the-art methods and exhibits superior in-domain and cross-domain transferability. IAT has been successfully deployed in real-world industrial recommender systems, including e-commerce advertising, shopping mall marketing, and live-streaming e-commerce, delivering substantial improvements in key business metrics.