Mixture-weighted models improve offline ad policy evaluation without propensities
Beyond IPS: Reliable Counterfactual Evaluation in Multi-Stage Ad Systems without Logged Propensities
Information Retrieval
Summary
Evaluating ad display strategies without knowing how likely previous ads were shown is tough. The authors study ways to estimate how good new ad policies are using logged data without needing these likelihoods. They introduce a method that adjusts the data to better match candidate policies and decides when offline estimates are reliable or when live tests are needed. Their approach performs better than older methods in simulations and in real ad systems, especially when data overlap is small.
What this means in practice
- •For online advertising engineers: Improve accuracy in selecting the best ad-ranking policies offline when the propensity data is missing, reducing reliance on costly live tests.
- •For recommendation system developers: Apply weighted reward modeling to offline evaluate ranking strategies when choice probabilities are unknown, enhancing policy selection quality.
Authors
Mohsen Malmir, Mohamed A. Radwan, Houssam Nassif, Murat Ali Bayir
Abstract
Offline evaluation of ad-ranking policies in multi-stage delivery systems is challenging because action propensities are often unavailable, limiting inverse propensity scoring (IPS). Reward-model-based estimators such as the Direct Method (DM) avoid propensities but can be biased when candidate policies induce impression distributions that differ from the logged data. We formalize DM-based counterfactual replay and propose mixture-weighted reward modeling, which reweights logged data toward a mixture of candidate-policy distributions using density ratios from a multiclass domain classifier. We bound each candidate's value-estimation error by overlap, cold-start, and replay terms, and derive an asymptotic rule that uses the cold-start mass to decide whether to trust offline selection or route candidates to online A/B testing. On an ads-delivery simulator and on production traffic from a large-scale CTR ranking system, the mixture-weighted reward model improves policy selection over its unweighted counterpart in both settings, with the largest gains at low overlap in simulation. Combined with a doubly robust correction, it attains the highest Top-1 accuracy in production and at low and medium overlap in simulation.