Time aware rotary position embedding improves recommendation accuracy
T-RoPE: Time-Aware Rotary Position Embedding for Sequential Recommendation
Artificial IntelligenceInformation RetrievalMachine Learning
Summary
Recommendation systems that suggest items like products or content often rely on the order of previous interactions but ignore the exact timing between them. The authors noticed that using only the order misses important signals like how much time passed or seasonal patterns. They created T-RoPE, a method that adds timing information to better understand user behavior. This improved recommendation accuracy on several datasets and even showed small but positive effects in a real online shopping app.
What this means in practice
- •For e commerce platform engineers: Enhance product recommendation quality by incorporating time-aware position embedding that captures elapsed time and seasonal user behaviors in large-scale systems.
- •For music streaming service developers: Improve playlist generation and user suggestions by using time-aware embeddings to reflect user engagement cycles and temporal patterns.
Authors
Yang Liu, Noel Loo, Ali Khanafer, Shuying Sun, Akshay Soni, Zhong Wu, Linjun Yang
Abstract
Large-scale recommenders increasingly adopt the sequential generative recipe behind large language models, bringing the Transformer into recommendation along with design choices made for text, including Rotary Position Embedding (RoPE). In language models, RoPE encodes token indices for relative position reasoning, but in recommendation, an interaction index records only event order, saying nothing about elapsed time, behavioral cycles across scales, or calendar phase. We revisit this choice and propose T-RoPE, a time-aware RoPE for sequential generative recommendation that replaces index-only rotation with timestamp-based angles, learnable temporal coefficients, multiscale frequency banks, shifted query alignment, and non-stationary key rotation. We prove that standard RoPE, even on timestamps, remains time-translation invariant and cannot distinguish seasonal contexts, and that T-RoPE breaks this invariance while preserving the RoPE interface. Across five public benchmarks, T-RoPE achieves the best result on every metric on every dataset, improving over the strongest baseline by 78--130\% in HR@10 on the sparse PixelRec data and 8--12\% across metrics on Amazon Books. On an industrial-scale e-commerce dataset with more than 6B interactions, it improves every metric over the HSTU + Time RAB backbone by 13--82\%, with ablations attributing the largest gains to multiscale frequencies ($+56\%$ NDCG@50) and non-stationary keys ($+4\%$). An online A/B test in the Shop app yields positive lifts in conversion rate ($+0.33\%$) and order count ($+0.63\%$). We also provide forward and backward algorithms whose added cost is linear in sequence length and head dimension, keeping time-aware RoPE practical for large generative recommenders.