Semi supervised learning improves cold start by following data geometry
Follow the Geometry, Not the Model: Cold Start Semi-Supervised Learning
Machine LearningArtificial Intelligence
Summary
Semi-supervised learning often struggles when only very few labeled examples are available because the model relies on its own predictions to learn. The authors propose a new method called VAST that first examines the shape and structure of the data using fixed features, then guesses labels for unlabeled examples based on this structure before training a classifier. This approach avoids the problem of the model supervising itself too early and performs better than previous methods on several datasets. It also produces a ready-to-use classifier without needing complex graph computations at test time.
What this means in practice
- •For machine learning engineers: Train classifiers with very limited labeled data by using geometry-based pseudo-label inference to improve performance.
- •For computer vision developers: Develop image classification systems that start effectively with few labels using VAST’s decoupled label inference and classifier training.
Authors
Itai David, Daphna Weinshall
Abstract
Modern semi-supervised learning (SSL) couples pseudo-label generation and classifier training, using the classifier's own confidence to select the pseudo-labels that are then used to update the model. In the cold-start regime, where at most a few labels per class are available, this coupling is ill-posed, since the classifier cannot supervise itself before it has learned. To address this problem, we propose VAST (Veracity-Aware Semi-Supervised Training), which decouples these two stages. Probabilistic beliefs over the unlabeled set are first inferred directly from the geometry of a frozen self-supervised embedding and only then distilled into an inductive classifier. The construction rests on the Veracity Matrix, a kernel-based structure that aggregates label evidence across the data manifold and admits an interpretation as a Dirichlet posterior under a per-observation powered-likelihood model. Additionally, we introduce Veracity Propagation, a self-terminating belief-spreading step that extends coverage beyond the kernel neighborhood of the labeled set. Under a controlled protocol in which all methods receive identical frozen embeddings and labeled sets, VAST outperforms the strongest graph-based SSL baselines at every operating point across three datasets, with statistically significant gains in 7 of 9 comparisons, while producing a deployable inductive classifier rather than requiring transductive graph inference. Compared with end-to-end confidence-gated SSL, we further find that these methods underperform in this setting and, in our experiments, do not consistently exceed labeled-only performance.