Dense features and optimization outperform sequence modeling for large-scale conversion prediction
Dense Feature Representation over Sequence Modeling: A Solution to the KDD Cup 2026 UniRec Challenge
Information Retrieval
Summary
Predicting whether people will take an action after clicking online ads is important but challenging. The authors worked on a big data challenge with tens of millions of records. They found that using dense feature representations and better optimization helped improve prediction accuracy more than complex sequence modeling techniques. Their careful testing showed that simpler approaches worked best for this task at large scale.
What this means in practice
- •For advertising platform engineers: Implement dense feature representations and orthogonalized optimization to improve large-scale click-to-conversion models for better ad targeting.
- •For e-commerce data scientists: Use dense feature techniques instead of fine-grained sequence models to increase accuracy of conversion prediction on massive customer interaction logs.
Authors
Yi Zhang, Weiliang Ji
Abstract
We describe our 10th-place solution to the KDD Cup 2026 Tencent UniRec Challenge, industrial click-to-conversion (CVR) prediction over 34.82M records, and we ask which mechanisms actually move held-out AUC. Starting from the official PCVRHyFormer baseline, a 15-step single-variable chain raises test AUC from 0.813237 to 0.827816, and our final submission reaches 0.828535. A leave-one-out ablation from the full model attributes the gain: removing the dense-feature representation stack costs 0.0095 AUC and removing the orthogonalized optimizer costs 0.0028, while no sequence-modeling component (merged single-stream backbone, polarity channel, auxiliary head, per-token FFN) costs more than 0.0005, within or adjacent to a $\pm$0.0004 seed band. We also report a generalization hazard: the row-group train/validation split shares one time window, so validation AUC overstates the leaderboard by about 0.014; anti-memorization and high-cardinality-ID changes even invert sign against it, a divergence that traces to dump-to-dump distribution shift and survives a time-ordered re-split. Dense representation and optimization, not finer sequence modeling, drive CVR AUC at this scale, and verdicts must come from the held-out leaderboard.