Revisiting Active Sequential Prediction-Powered Mean Estimation
2026-04-20 • Machine Learning
Machine Learning
AI summaryⓘ
The authors studied a method for estimating averages by deciding when to ask for the true label of data points and when to trust predictions from a machine learning model. They found that relying less on the model's uncertainty and more on a fixed query probability often gave more reliable confidence intervals. They then analyzed this method mathematically and showed that the query probability tends to settle at a fixed value when using a learning strategy that adapts over time. Their computer simulations supported these theoretical findings.
active sequential predictionmean estimationquery probabilityconfidence intervaluncertainty samplingnon-asymptotic analysisno-regret learningmachine learning modelscovariatessimulation
Authors
Maria-Eleni Sfyraki, Jun-Kun Wang
Abstract
In this work, we revisit the problem of active sequential prediction-powered mean estimation, where at each round one must decide the query probability of the ground-truth label upon observing the covariates of a sample. Furthermore, if the label is not queried, the prediction from a machine learning model is used instead. Prior work proposed an elegant scheme that determines the query probability by combining an uncertainty-based suggestion with a constant probability that encodes a soft constraint on the query probability. We explored different values of the mixing parameter and observed an intriguing empirical pattern: the smallest confidence width tends to occur when the weight on the constant probability is close to one, thereby reducing the influence of the uncertainty-based component. Motivated by this observation, we develop a non-asymptotic analysis of the estimator and establish a data-dependent bound on its confidence interval. Our analysis further suggests that when a no-regret learning approach is used to determine the query probability and control this bound, the query probability converges to the constraint of the max value of the query probability when it is chosen obliviously to the current covariates. We also conduct simulations that corroborate these theoretical findings.