Robust policy learning improves decisions when future data changes
Distributionally-Robust Policy Learning with Observational Data
Machine Learning
Summary
Sometimes policies, like deciding who gets benefits, are made using old data that might not match new cases. The authors study how to learn policies that work well even when the future data is somewhat different from what was seen before. They propose a method that balances caution and effectiveness by considering the worst-case scenarios within a certain range of differences. Their approach mathematically guarantees good learning performance and works well in simulated tests based on real-world problems. This helps make better decisions from observations without relying on perfect past data.
What this means in practice
- •For policy analysts: Develop policies that maintain effectiveness when the target population or environment shifts after data collection.
- •For healthcare data teams: Design treatment recommendation systems that stay reliable despite changes in patient populations or healthcare practices.
Authors
Debmalya Mandal
Abstract
Policy learning from observational data is fundamental to applications ranging from personalized medicine to welfare-program targeting, but standard approaches optimize empirical performance and may fail when the deployment distribution differs from the training distribution. We study \emph{distributionally-robust policy learning}: given observational data, we seek a policy $π\in Π$ that maximizes the worst-case policy value over an $f$-divergence ball of radius $ρ$ around the data-generating distribution. We propose an estimator built on cross-fitted doubly-robust scores for the conditional treatment effect, paired with a DRO solver that handles \texttt{KL}, $χ^2$, and \texttt{CVaR} ambiguity sets. We establish two convergence guarantees. First, the estimated policy attains the standard $\widetilde{O}(n^{-1/2})$ rate on its DRO regret, and inherits the doubly-robust property -- the rate continues to hold whenever either the outcome model or the propensity model is consistently estimated, even at slow nonparametric rates. We then propose a sample-splitting bias-correction algorithm derived from the Lagrangian dual, and show that under a Tsybakov-style margin condition on the treatment-effect, the estimator attains a fast rate of $\widetilde{O}(n^{-(1+α)/(2+α)})$ , interpolating between $\widetilde{O}(1/\sqrt{n})$ and $\widetilde{O}(1/n)$ as the margin parameter $α$ ranges over $[0,\infty)$. To our knowledge, this is the first work to incorporate a margin condition into the analysis of distributionally-robust policy learning, and the first to obtain fast rates in this setting. Empirically, on synthetic and semi-synthetic experiments calibrated to a welfare-to-work program, distributional robustness yields meaningful improvements over empirical welfare maximization when training and test distributions differ.