Kernel weighted importance sampling for off-policy evaluation in contextual bandits
Machine Learning
Summary
The authors introduce a new method called Kernel-WIS to estimate how well different choices would do, using only offline data in situations called contextual bandits. They show that this method gets more accurate as it uses more data and works better than other common methods, especially when the underlying assumptions about past behavior are wrong. Their approach cleverly combines two existing sampling techniques to get more reliable results. This helps evaluate decisions without needing new experiments.
Authors
Joshua Spear, Matthieu Komorowski, Rebecca Pope, Neil J Sebire, Erica E. M. Moodie
Abstract
This article presents a novel estimator for performing off-policy evaluation using only offline data for contextual bandits. The proposed estimator, Kernel-WIS is demonstrated to be asymptotically consistent and to empirically outperform strong baselines (including vanilla weighted importance sampling), particularly under complex conditions including behaviour policy miss-specification. The benefit of Kernel-WIS is derived from combining the bounded property of vanilla weighted importance sampling with the linearity of vanilla importance sampling.