Low budget active learning improves model training on small data sets
Low-Budget Active Learning through Entropic Optimal Transport
Machine Learning
Summary
Training models often needs lots of labeled examples, which can be expensive especially when experts must label data, like in medical tests. The authors developed a method to choose a small, smart subset of data points, called a coreset, that is good enough to train accurate models. They use a math tool called entropic optimal transport to pick these points efficiently and with good quality. Tests show this method works better than common alternatives when only a few labeled samples are allowed.
What this means in practice
- •For medical data teams: Select small high-value subsets of medical images for expert labeling to train accurate diagnostic models under limited budgets.
- •For machine learning engineers: Improve model training efficiency on costly labeled datasets by optimizing sample selection using entropic optimal transport.
Authors
Rim Hajal, Mathieu Besançon, Jérôme Malick
Abstract
We consider low-budget active learning, which consists of selecting a limited number of points, the coreset, such that a model can be trained to high accuracy on the selection only. This problem is particularly relevant in contexts where labeling requires costly expert intervention, as in medical applications. We leverage features extracted from a pretrained self-supervised model to represent the data, and perform coreset selection directly in this feature space. In this paper, we use entropic optimal transport, specifically the Sinkhorn divergence, as the coreset selection criterion, which first allows us to get dimension-free sample complexity results, and second admits computationally efficient gradient evaluations. This opens the way to using gradient-based algorithms to rapidly compute solution candidates, further improved by a swap-based local search, with guarantees on the solution quality. Experiments on image benchmarks and medical datasets show that our method outperforms state-of-the-art heuristics in low-budget settings.