Data release must keep key action weights for accurate policy evaluation
What a Data Release Must Preserve for Off-Policy Evaluation
Machine Learning
Summary
Evaluating the impact of a policy using existing data can become impossible if important differences in the data are lost when it is shared. The authors explain exactly what details must be preserved in data releases to ensure reliable evaluation of how well a policy works. They offer methods to organize data that keeps these key details and to improve existing data so evaluations remain trustworthy. Their techniques greatly reduce uncertainty when using released data for policy assessment in recommendation systems.
What this means in practice
- •For data engineering teams: Create data releases that preserve essential policy evaluation weights to ensure off-policy evaluation remains valid.
- •For recommendation system engineers: Reduce uncertainty in evaluating recommendation policies by applying data repair methods that optimize released data grouping.
Authors
Kyungbok Lee, Yating Zou, Michael R. Kosorok
Abstract
In off-policy evaluation, merging decision-time histories that matter differently for a specified policy value or comparison can make that evaluation unidentified, even with observed actions and outcomes and known behavior and target policies. We characterize which distinctions must be preserved: identification holds if and only if the action-specific evaluation weights are constant within each released record, and the coarsest exact release groups histories by these weights. For normalized outcomes, the largest sharp width induced by the release equals a behavior-weighted within-record radius of these weights. The same radius becomes a release-design objective when each record may be refined into at most $m$ categories. The optimum has an exact $m$-center characterization. When behavior does not vary within a released record, we give a finite-data repair algorithm whose maximum evaluation-weight radius is at most twice the finite-data optimum. For a fixed action-outcome distribution within a released record, a distinct sharp-identification problem remains. Under the same condition, the action-specific outcome distributions are identified, and optimizing over their couplings gives the sharp interval. We construct an AIPW envelope containing this sharp interval; it matches the interval for all such outcome distributions exactly when the extreme within-record action-coefficient vectors take at most two values in each action coordinate. In 24 KuaiRec tasks, the repair algorithm yields a 93% median reduction in mean pairwise sharp width at $m=64$ relative to the unrepaired release.