QueryFormer improves click conversion predictions with efficient query attention
QueryFormer: Winning Solution for KDD Cup 2026 Tencent UniRec Challenge
Artificial Intelligence
Summary
Predicting whether someone will take an action after clicking an ad or link requires understanding both their past behavior and many different factors together. The authors noticed existing methods don’t refine how they focus on key information when combining these details. They created QueryFormer, which uses an attention mechanism to better connect diverse information and user behavior sequences efficiently. This approach helped them win a major industry challenge by improving prediction accuracy while keeping model latency low.
What this means in practice
- •For online advertising teams: Enhance conversion rate predictions by integrating diverse user features and behavior sequences with an efficient attention-based model.
- •For e-commerce recommender engineers: Use the unified field-sequence block to better combine product features and user actions for improved recommendation accuracy with controlled latency.
Authors
Yuanzhe Zhou, Zhaoyang Zeng
Abstract
Post-click conversion rate (pCVR) prediction requires jointly modeling feature interactions and sequential user behaviors. The KDD Cup 2026 Tencent UniRec Challenge calls for a unified architecture addressing both. We observe that existing unified architectures often generate query tokens---the central information hub---with projection-based multi-layer perceptrons (MLPs), without explicit token-to-query attention for refining the query side. We propose QueryFormer, centered on a stackable unified field--sequence block that bridges non-sequential multi-field features and behavioral sequences, and provide a latency-aware scaling study over view width $H$, model width, depth, data, and compute. The block generates queries through cross-attention and packs sequence queries into shared-parameter attention. QueryFormer secured 1st place in the Industrial Track, achieving an official test area under the ROC curve (AUC) of 0.83254; a modest post-competition scale-up reached 0.832713. Within our grid, $H$-scaling improves validation AUC from 0.84540 to 0.84615 and beats HyFormer at comparable budgets. Ablation identifies query generation as the largest contributor. Packed shared-parameter cross-attention keeps H=8 inference latency to only 1.89x that of H=1, positioning the bridge as an efficient stackable unified block.