Policy learning balances expected welfare and sampling risk with stability
Stable Policy Learning
Machine Learning
Summary
Making good policy decisions based on data from experiments can be tricky because the results might change if someone else takes part in the experiment. This randomness means that even good policies might sometimes perform poorly. The paper shows that if the policy recommendations don’t change much when one person in the data is swapped out, then those recommendations are more reliable. To do this, the authors propose a way to generate policies by combining decisions from many smaller groups within the experiment, which makes the recommendations more stable and better for cautious decision makers.
What this means in practice
- •For public policy analysts: Generate more reliable policy recommendations from experimental data that balance good outcomes and reduced risk due to sample changes.
- •For healthcare program designers: Develop treatment assignment strategies that maintain stable welfare outcomes when experiment participants vary, improving risk management in health interventions.
Authors
Harvey Barnhard, Giacomo Opocher, Rahul Singh
Abstract
In evidence-based policymaking, typically one experimental sample is observed, then a learned policy recommendation is implemented at scale. Policies learned from the experimental data can perform well in expected welfare, yet random sampling in the experiment can produce recommendations with poor welfare outcomes. In this paper, we ask: how should policy learning algorithms balance expected welfare against sampling risk? Our main contribution is to show that algorithmic stability plays a central role in characterizing and navigating the tradeoff. Intuitively, if a policy learning algorithm's recommendation remains stable when one experimental unit is replaced, then that algorithm has limited sampling risk. We propose a method for policy learning called policy-vote bagging, which learns treatment decisions on many subsamples then averages their votes into treatment probabilities. Relative to using one subsample, averaging across subsamples preserves expected welfare and improves expected utility for a risk-averse researcher. We derive sharp bounds linking estimation accuracy, subsample size, and welfare variation, including an exact guarantee under CARA utility.