Papers for

online education platform engineers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Off-policy evaluation improved with budgeted multi-source counterfactual annotations

Budgeted Multi-Source Counterfactual Annotation for Off-Policy Evaluation

Abstract: Off-policy evaluation (OPE) estimates the value of a target policy from logged data, but limited behavior-policy coverage can force high-variance reweighting or reward-model extrapolation. Counterfactual annotations can add evidence about unobserved actions, yet practical sources, including domain experts and large language models (LLMs), may be costly, biased, or noisy. We study budgeted acquisition of such annotations for contextual-bandit OPE. Given source-specific costs and error profiles, we formulate an integer allocation problem over context-action pairs and annotation sources to minimize the component of estimator variance that depends on the annotation plan. We characterize when annotations are valuable through a first-annotation threshold and local annotation-value regimes. For the coupled multi-source problem, we develop a majorization-minimization algorithm with dynamic-programming subroutines that monotonically improves the objective. Experiments in synthetic clinical and LLM-annotated education bandits show that our allocation method reduces fixed-profile mean squared error (MSE) by 20.58% and 10.77%, respectively, relative to no annotation.

Wed 7 OctMachine LearningArtificial Intelligence
The gist
Off-policy evaluation is a way to predict how well a new decision-making strategy might work using old data. But often that old data doesn’t cover all decisions well, which makes predictions shaky. The authors show that adding extra information about what would have happened if other choices were made can help, but getting this extra info can be expensive or noisy. They develop a way to decide how to spend a limited budget on different kinds of extra information to reduce errors in evaluating strategies. Their method lowers prediction errors in simulated healthcare and teaching scenarios.
Open → 2610.10974v1