Papers for

clinical trial managers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Pro-active inference improves accuracy in costly data sampling

Pro-Active Inference

Abstract: Estimating a population mean is fundamental across scientific applications, but observing every outcome can be prohibitively expensive. Often, covariates and machine learning (ML) predictions are available for the full population, even when only a small subset of outcomes can be observed. How can we leverage this information to proactively select observations to label, update ML predictions as labels arrive, and construct statistically valid confidence intervals? We develop a general, without-replacement formulation of Pro-Active Inference (PAI), extending our earlier with-replacement approach and building on Active Statistical Inference. Rather than making fixed-order label-skip decisions, PAI proactively selects which observation to label next from the remaining covariate pool while accommodating online ML updates. We establish PAI's finite-sample unbiasedness and asymptotic validity, develop an improved finite-sample variance estimator, and extend PAI to superpopulation inference without Bonferroni corrections or $α$-splitting. We characterize PAI's potential efficiency gains over Batch Active Inference in a quasi-oracle setting, highlighting the role of prediction-residual imbalance. Across eight diverse datasets, PAI achieves greater sampling efficiency than Sequential Active Inference in the vast majority of tested settings with near-nominal statistical coverage, using the same ML predictor and uncertainty score. However, comparisons with simpler baselines such as Prediction-Powered Inference and uniform sampling, as well as the closely related Prediction-Powered Active Testing, reveal when additional adaptivity helps or hurts statistical efficiency. Through interpretable, theory-motivated diagnostics and frozen-ML ablations, we investigate when and why PAI gains or loses efficiency, highlighting the initial residual-estimation problem and subsequent online learning dynamics.

Mon 26 OctMachine Learning
The gist
Measuring the average outcome in a population is important but often expensive because you can't collect every data point. The authors introduce a method called Pro-Active Inference (PAI) that smartly picks which data points to label next by using existing predictions and information from the whole group. This approach updates predictions on the fly as new labels come in and aims to give precise estimates without wasting resources. Tests on multiple datasets show it usually outperforms previous methods, revealing when its adaptability helps or sometimes even hinders accuracy.
Open → 2610.34681v1