AI analysts keep mistakes low while getting better over time
Lifetime-valid decisions by continually evolving AI analysts
Artificial IntelligenceMachine Learning
Summary
Deciding when AI systems make errors is tricky because these systems can change and improve continuously. The authors propose a method called continual validation, which tests every new AI version on fake data to make sure it doesn’t make too many wrong decisions. This method keeps the risk of mistakes in check throughout the AI’s lifetime, no matter how often the AI changes. At the same time, it allows the AI to get better without losing reliability, shown in examples like medical studies and product testing.
What this means in practice
- •For clinical trial teams: Ensure evolving AI tools maintain low false approval rates when analyzing drug equivalence studies.
- •For product experiment teams: Validate updates to AI analysts in A/B testing to keep false positive rates under control while improving decision accuracy.
Authors
Xiaojing Sun, Yuhan Zeng, Xiao Wang
Abstract
AI agents increasingly act as analysts whose decision can often be cast as hypothesis tests. Classical tests control the Type I error rate, the probability of rejecting a true null hypothesis, for fixed procedures. However, AI analysts introduce internal randomness and can evolve continuously over time. As AI analysts evolve, the risk of approving versions with Type I error rates above the target can accumulate, and adaptive use of evaluation feedback can lead to overfitting. We introduce continual validation, which evaluates each proposed AI analyst version before deployment using simulated data under the null hypothesis, with scenarios that can be randomized and expanded over time. We prove that, regardless of how updates are chosen or how many are proposed, the lifetime probability of ever deploying a version whose Type I error rate exceeds the allowed level remains bounded. This guarantees that every subsequent decision controls the Type I error rate at the target level across these scenarios. At the same time, updates with sufficiently low Type I error rates pass validation with high probability, allowing the analyst to continuously improve its performance without sacrificing rigor. In two examples involving evolving AI analysts for bioequivalence studies and product A/B tests, Type I error rates reached 0.098 and 0.183 without continual validation, compared with at most 0.045 and 0.044 with it, while retaining 82% and 96% of the power from optimally selected valid updates.