Stochastic optimization trains models with mixed real and AI labels
Training with ML-labeled Data via Stochastic First- and Zeroth-Order Optimization
Machine Learning
Summary
Training AI models usually needs lots of examples with correct answers, which can be expensive and slow to get from humans. This paper looks at training models using some real labels and some labels produced by another AI, which might have mistakes. The authors propose new algorithms that can learn effectively even when some labels are automatically generated and possibly wrong. They test these methods on simple and complex models, including large language models, showing the approach works well.
What this means in practice
- •For machine learning engineers: Train models using a mix of verified and automatically generated labels efficiently to save human labeling costs.
- •For data pipeline developers: Implement training pipelines that combine real and ML-predicted labels robustly, improving scalability for large datasets.
Authors
Maria-Eleni Sfyraki, Jun-Kun Wang
Abstract
Training a machine learning (ML) model is an optimization process that typically requires a set of examples with ground-truth labels beforehand. Obtaining these ground truths usually requires laborious human effort and can be costly. The scalability of human labeling becomes a concern when there are a large number of data points that need to be labeled. To address this practical issue, we consider an optimization setting where a subset of samples has ground-truth labels, while the remaining data points were labeled by an ML predictor, which could potentially assign incorrect labels to those points. We propose both first-order and zeroth-order stochastic optimization methods for training on datasets consisting of a mix of true-labeled and ML-labeled samples. We further conduct comprehensive empirical studies from training a linear regression model to fine-tuning a large language model (LLM) to showcase the effectiveness of the proposed algorithms for training with ML-labeled data.