Online treatment selection controls errors while learning from limited feedback

Online Individualized Treatment Selection under Partial Feedback: False Discovery Control and Minimax Optimality

Machine Learning

Summary

When deciding who should get a treatment based on their personal characteristics, we often only see the result for those who actually get the treatment, but not what would have happened otherwise. This paper studies a method called PAITS that learns over time to pick treatments for individuals while controlling mistakes where the treatment is not actually better. The method starts with existing data, tries some options randomly, and updates how it chooses treatments based on new results. The authors prove that PAITS balances learning well and making few errors, and show it works using simulations and one real dataset.

What this means in practice

  • For clinical trial designers: Design trials that adaptively assign treatments to patients while controlling the rate of ineffective or harmful treatments.
  • For online recommendation engineers: Develop recommendation systems that identify which users benefit from an option with limited feedback while minimizing wrong recommendations.

Authors

Runzhe Gu, Zinan Zhao, Wenguang Sun, Xintao Xia

Abstract

We study online individualized treatment selection when only the outcome under the assigned treatment is observed. Our objective is to preferentially treat individuals whose potential outcomes under treatment exceed a covariate-specific control reference, while limiting false discoveries among treated individuals. We propose Partial-Feedback Adaptive Individualized Treatment Selection (PAITS), which initializes learning from historical data and combines sequential estimation of local null risks, randomized exploration, and feedback-driven updates of the treatment selection threshold. Under regularity conditions, we establish asymptotic false discovery rate control and sublinear gap-weighted assignment regret relative to a population oracle. We further establish a minimax lower bound on assignment regret under prescribed uniform nuisance-estimation requirements and give conditions under which PAITS attains the corresponding constrained minimax rate up to logarithmic factors. These results quantify the tradeoff between accurate treatment assignment and nuisance learning under partial feedback. Simulation studies and a real-data analysis using the Jester recommendation dataset illustrate the finite-sample performance of PAITS.