Summary
Long histories of user behavior help recommendation systems give better suggestions, but looking at all past actions at once is slow. The authors studied how attention spreads over these histories and found it shrinks to key past actions over time and stays stable between steps. They designed a system called Progressively Focused Attention (PFA) that first looks broadly, then narrows attention to the most important past actions, reducing the work needed while keeping important details. Tests on public and industrial datasets showed PFA is faster and better, with real-world experiments confirming it improves e-commerce ad performance without slowing down recommendations.
What this means in practice
- •For e-commerce platform engineers: Implement PFA to reduce online latency and improve recommendation quality for users with long interaction histories.
- •For advertising technology teams: Use PFA to efficiently serve personalized ads by focusing computation on critical past user behaviors, boosting engagement without extra delay.
Authors
Pengjie Zhou, Weiqin Yang, Haibo Xing, Hao Deng, Jinxin Hu
Abstract
Long user behavioral sequences improve the performance of generative recommendation (GR), but computing full attention over long histories during autoregressive beam search decoding incurs substantial online inference latency. To accelerate long-sequence decoding, existing approaches typically rely on static history compression or dynamic top-k sparse attention; however, the former risks losing critical semantics, whereas the latter incurs significant overhead from repeated token selection and non-contiguous memory access. In this work, we thoroughly analyze attention patterns during GR decoding and reveal two key properties: (i) progressive attention focusing, where attention progressively focuses on a small subset of critical historical tokens; and (ii) cross-step ranking consistency, where their importance rankings remain relatively stable across subsequent decoding steps. Motivated by these observations, we propose Progressively Focused Attention (PFA), an efficient sparse decoding framework for long-sequence GR. PFA retains full attention at the initial decoding step to explore global historical preferences. It then establishes a shared importance ranking through cross-beam weighted aggregation and reorders the key-value cache of the selected tokens into a contiguous memory layout. Subsequent decoding steps reuse this ranking and geometrically reduce the token budget, preserving critical semantics while lowering computational and memory access overhead. Extensive experiments on two public benchmarks and a large-scale industrial dataset demonstrate that PFA outperforms state-of-the-art baselines. A 7-day online A/B test on a large-scale e-commerce advertising platform further confirms that PFA delivers substantial business gains while maintaining low inference latency. Our code is publicly available at https://anonymous.4open.science/r/PFA-review-088D.