Geometric consistency enables label-free checkpoint selection in audio active learning
Geometric Consistency Based Label-Free Validation for Checkpoint Selection in Audio Active Learning
SoundArtificial IntelligenceMachine Learning
Summary
Labeling audio data for training classifiers can be expensive and slow because it requires humans to tag audio clips. The authors present a way to pick the best saved model checkpoint during training without needing extra labeled data for validation. They use the shapes and patterns in the unlabeled audio data, predicted by simple classifiers trained on the labeled data, to score each checkpoint. This method saves labeling effort while keeping good performance and works with various training strategies.
What this means in practice
- •For audio machine learning engineers: Choose the best model checkpoints during audio classifier training without needing extra labeled validation data, reducing labeling costs.
- •For speech recognition developers: Improve training efficiency by validating audio models using geometry-based scores on unlabeled audio data, compatible with multiple selection strategies.
Authors
Shiqi Zhang, Tuomas Virtanen
Abstract
Audio active learning (AL) trains a classifier at low annotation cost by labeling only an informative fraction of an unlabeled collection over successive rounds. Each round picks one of the checkpoints saved during training. Mainstream AL protocols do so with an assumed validation set, but in practice its labels also count toward the annotation cost, shrinking the advantage of an already small annotation budget. We propose geometric consistency, a label-free validation rule for checkpoint selection. Both classifiers, nearest-class-mean and k-nearest-neighbor, are built from the labeled set in a frozen pretrained embedding space. They then predict the unlabeled data to model its geometry. The rule scores each checkpoint by its agreement with these pseudo-labels and its cross-entropy on the labeled set. It works with any query strategy, has one hyperparameter, insensitive in practice, and adds only one batched forward pass over the unlabeled data per round. At matched label budgets across datasets, encoders, and query strategies, it attains the best mean rank in area under the labeling curve (AULC) among methods without extra labels and is on par with an out-of-budget validation set. To our knowledge, this is the first systematic study of label-free validation for checkpoint selection in audio AL.