Estimating sample size for machine learning using local linear models
A Statistical Approach to Estimating Sample Size of Machine Learning Models
Machine LearningArtificial Intelligence
Summary
Figuring out how much data is needed to train machine learning models is tricky because traditional methods require knowing details about how inputs relate to outcomes beforehand. Many machine learning models are complex and don’t fit simple formulas, making those methods unusable. The authors offer a new way to estimate sample size by breaking down complex models into simpler pieces that look like straight lines in small areas. This lets them check how much data is needed in each piece and combine the results to get an overall estimate.
What this means in practice
- •For machine learning engineers: Estimate necessary data amounts to confidently train complex machine learning models without oversampling.
- •For clinical data scientists: Plan data collection for predictive models in healthcare where outcomes are rare or relationships nonlinear.
Authors
Dat Phan-Trong, Sunil Gupta, Svetha Venkatesh
Abstract
Sample size determination for machine learning (ML) prediction models is challenging because conventional power analysis typically requires the predictor-outcome relationship and effect structure to be specified a priori. Nonlinear ML models learn complex prediction surfaces that do not admit straightforward analytical power calculations. We propose a framework that approximates nonlinear ML models with localized linear representations and estimates sample size requirements by evaluating statistical power across these local regions.