Transformers improve click prediction with better position and timing models
Rethinking Positional Encoding and Interaction Granularity for CTR Prediction
Information Retrieval
Summary
Predicting whether someone will click on an item depends on understanding the order and timing of their past behaviors. The authors found that common ways to mark position and time in these predictions don't perform better than ignoring them. They created TRIP, which better captures each behavior's place in history and its time relative to the target item. Their method groups behaviors by time and models interactions within and between groups more flexibly, leading to better click predictions.
What this means in practice
- •For advertising platform engineers: Improve click-through rate models by using target-aware position and grouped temporal interaction to better predict user clicks.
- •For e-commerce recommendation teams: Enhance product recommendation accuracy by more precisely encoding customers' behavior sequences and temporal relations.
Authors
Xinxun Zhang, Yuxiang Wang, Xiaokai Zhou, Jiayue Li, Yi Li, Yue Liu, Jun Zhang, Xiaoling Zhou, Tieyun Qian, Xiao Yan, Jiawei Jiang
Abstract
Click-through rate (CTR) prediction relies on historical behaviors and contextual features to estimate clicks on a target item. Existing Transformer-based CTR models often adopt index-based or temporal positional encodings and token-level causal interaction. However, we observe that most existing positional encodings do not outperform NoPE, while relaxing causal constraints within target-relative temporal groups also yields gains. These observations motivate a fundamental question: what positional encoding and interaction granularity does CTR prediction actually require? To address this question, we propose \textbf{TRIP}, which combines target-aware rank--time positional encoding with temporal-group interaction modeling. Specifically, Factorized Rank Encoding represents each behavior's ordinal position within the historical sequence through a low-rank factorization, while Target-Relative Time Encoding characterizes its temporal distance to the target on a coarse-to-fine scale. This joint design grounds each behavior's positional representation in its sequence context and temporal relation to the current target. Furthermore, Temporal Group-Causal Interaction enriches intra-group representations through bidirectional interaction and masked reconstruction while preserving inter-group causal order. Finally, behavior representations are condensed into compact group summaries for efficient group-level modeling in subsequent layers. Experiments on four public benchmarks show that TRIP improves AUC by up to 4.45\% relative to the strongest baseline.