Papers for

advertising platform teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Knowledge distillation improves click conversion predictions in recommendation systems

Serving-Aligned Entire-Space Conversion Modeling via Click-Through Rate Knowledge Distillation

Abstract: Recommendation systems may use predicted click-through conversion rate (pCTCVR), the product of predicted click-through rate (pCTR) and post-click conversion rate (pCVR), to guide ranking decisions. Many pCVR models build on the Entire Space Multi-Task Model (ESMM), which jointly trains pCVR and auxiliary pCTR towers using pCTR and pCTCVR losses, but deploys only the pCVR tower. At serving time, this tower may be paired with a separately trained pCTR model, creating a training-serving mismatch in pCTCVR when its predictions differ from those of the auxiliary pCTR tower. In this paper, we use knowledge distillation (KD) from logged serving pCTR scores to supervise ESMM's auxiliary pCTR tower and improve training-serving pCTCVR alignment. We denote variants by KD-$α$, where $α$ controls the hard-label loss weight. On the Ali-CCP dataset, KD-0.3, the best variant on serving pCTCVR, increases ROC-AUC not only for serving pCTCVR but also for pCVR by 2.1% and 1.6%, respectively, and PR-AUC by 9.9% and 8.8%, respectively, relative to the ESMM baseline. We also test negative downsampling with 5% nonclick retention. Downsampled KD-0.3 increases mean AUC for serving pCTCVR and pCVR by 1.2% and 0.9%, respectively, -and PR-AUC by 8.5% and 4.5%, respectively, relative to downsampled ESMM.

Wed 11 NovMachine Learning
The gist
Predicting how often users will click and then buy something is important for recommendation systems. The authors found that training models using information from the actual click predictions made during service (called knowledge distillation) helps make better and more consistent conversion rate predictions. This method leads to more accurate rankings of items that users are likely to click and purchase. Tests on a large e-commerce dataset show this approach improves prediction quality compared to previous methods.
Open → 2611.15137v1

Framework identifies causes of search quality problems end to end

WISE: An Agentic Framework for Whole-page Multimodal Search Experience Attribution

Abstract: Algorithm-driven services, including search, recommendation, and advertising, rely on multi-stage model pipelines. While research prioritizes model optimization, post-deployment monitoring focuses on operational reliability. Yet operationally healthy systems may still produce poor-quality results, exposing a gap in existing evaluation and monitoring practices. However, a systematic problem formulation and benchmark for this task remain lacking. To bridge this gap, we formulate System Quality Diagnosis (SQD) for algorithm-driven services, a new task that extends quality evaluation beyond identifying poor outcomes to attributing their causes across system components. Focusing on industrial search, we propose WISE, an agentic framework for end-to-end quality diagnosis that discovers user-facing issues and traces their origins to underlying algorithmic components. To improve diagnostic reliability, we introduce ReTrace for constructing evidence-consistent trajectory supervision and Adaptive Harness-Policy Optimization (AHPO) for failure-guided optimization of the agent harness and model policy. To enable systematic evaluation, we construct WISE-BENCH, a real-world multimodal search benchmark that combines user-facing evidence, internal pipeline traces, and expert diagnosis annotations. Extensive experiments show consistent gains across model scales, with WISE achieving 75.20% end-to-end accuracy on Qwen3.5-35B-A3B. In production, WISE supports quality auditing, online quality assurance, and model optimization, reducing diagnosis costs and improving downstream relevance models.

Wed 4 NovInformation Retrieval
The gist
Sometimes search engines or recommendation systems work fine but still give poor results, and it's hard to know why. The authors introduce a way to track down where these problems come from inside multi-step systems. They created WISE, a tool that looks at the whole process, from what the user sees back to the computer models making decisions, to find where things go wrong. They also built a new set of test data combining real search results and expert labels to measure how well their tool works.
Open → 2611.06777v1

Multi task learning improves recommendation accuracy and satisfaction

Learned Cross-Task Relationships in Multi-Task Models

Abstract: We propose a framework that learns cross-task relationships in multi-task models by approximating the joint distribution of task labels through targeted pairwise relationships. This approach improves performance via transfer learning and enhances information extraction without the intractable complexity of modeling the full joint space. Although our framework applies to any multi-task system, we demonstrate its efficacy within YouTube's production recommendation systems. Experiments across the Notifications, Homepage, and Watch Next surfaces show improvements in both accuracy and user satisfaction metrics. Finally, we propose a workflow template to facilitate broader future implementation.

Wed 23 SeptArtificial Intelligence
The gist
Many online services try to do several related tasks at once, like recommending videos and sending notifications. The authors introduce a way to help computers understand how these tasks relate to each other by looking at pairs of tasks rather than everything at once. This approach makes the system better at learning and sharing useful information, improving recommendation performance. They tested their method on YouTube’s recommendations and found it made users happier with the suggestions they received.
Open → 2609.28776v1

Lightweight ranking heads speed up recommender system experiments

Lightweight Ranking Heads: Accelerating Multi-Task Experimentation in Production Recommender Systems

Abstract: Modern production-scale recommender systems rely on complex, multi-task ranking models. Introducing new prediction tasks into these massive systems often causes bottlenecks - it risks negative task conflicts with existing tasks, and can lead to long development and experimentation cycles due to the expensive retraining of backbone models and downstream models or tuning of reward combination formulas. To address the critical challenge of slow experimentation velocity, we introduce the Lightweight Ranking Heads (Light Heads) framework. Designed for continuous online learning environments, Light Heads enable the dynamic injection of new tasks into existing multi-task ranking models, effectively obviating the need for model cold-starting and retraining of backbone models. By utilizing stop-gradients and stateless daily training, this design strictly isolates new tasks, mitigating the risk of adverse task conflicts. Crucially, this framework uses a centralized configuration that allows Light Heads to be added to multiple models simultaneously, unblocking faster training data generation and co-training of downstream models. Successfully deployed at YouTube scale, this approach reduces the iteration cycle for multi-task experimentation from several weeks to days. In this paper, we detail the system architecture, analyze the training dynamics of stateless cold-started heads, compare their performance to full heads, and demonstrate how Light Heads have enabled the rapid A/B experimentation and deployment of new ranking tasks that yield measurable production value.

Mon 21 SeptMachine LearningArtificial Intelligence
The gist
Adding new tasks to big recommendation models is usually slow and tricky because it can interfere with existing tasks and requires retraining the whole system. The authors present Lightweight Ranking Heads, a way to add new tasks on the fly without retraining the main model. This method keeps new tasks separate and allows updating many models at once, making experiments much faster. It has been successfully used at YouTube, cutting experiment time from weeks to days.
Open → 2609.25433v1