Click-through prediction improved by combining multiple estimator models
Beyond Interaction Capacity: Estimator Scaling with Recursive Models for CTR Prediction
Machine LearningArtificial Intelligence
Summary
Predicting click-through rates helps websites show better recommendations and ads by understanding how features interact. The authors found that just making one prediction model bigger doesn’t keep improving accuracy as much anymore. Instead, using multiple related models together in a smart, efficient way can lead to better predictions. They created a method called RECAP that uses this idea without needing lots of extra resources. Tests show RECAP performs very well on standard prediction tasks while keeping the model size manageable.
What this means in practice
- •For advertising platform engineers: Deploy more accurate CTR prediction models that combine multiple estimators efficiently to improve ad targeting performance without a large increase in model size.
- •For recommendation system developers: Integrate parameter-efficient recursive models to enhance recommendation quality by leveraging diverse model predictions with limited additional computational cost.
Authors
Shivang Chopra, Fotis Iliopoulos, Zsolt Kira, Gaurav Menghani
Abstract
Click-Through Rate prediction, a core task in recommendation and advertising systems, relies on modeling interactions among sparse categorical features. Explicit cross networks are a central paradigm for CTR prediction, and recent progress has largely come from increasing the interaction capacity of a single predictor through deeper cross networks and more expressive cross operators. We revisit whether continually increasing interaction capacity remains the most effective way to improve predictive performance, and find that its benefits quickly exhibit diminishing returns even as capacity continues to grow. This motivates a complementary scaling direction that we call estimator scaling, where additional resources are used to incorporate multiple related estimators rather than only enlarging a single predictor. Through theoretical analysis, we show that the gains from estimator scaling are governed by the amount of non-shared predictive variation available across estimators. However, exploiting this variation naively can be expensive: independently trained models provide substantial estimator diversity but require deployment cost to grow with ensemble size. This motivates a parameter-efficient realization of estimator scaling that can incorporate diversity from multiple estimator sources without maintaining multiple full models. Building on this view, we introduce RECursive Averaged Predictor (RECAP), a parameter-efficient recursive CTR model that operationalizes estimator scaling at three levels: distillation across independently trained models, exponential moving averaging over training trajectories, and aggregation over inference-time routes within a weight-shared recursive backbone. Experiments across multiple benchmarks establish new state-of-the-art predictive performance on standard benchmarks, while placing the RECAP on a favorable performance-parameter Pareto frontier.