Semi-Supervised Conditional Diffusion via Label Augmentation
Machine Learning
Summary
The authors present a new method called label-augmented conditional diffusion (LACD) to improve how machines learn from labeled data by also using unlabeled data. They treat unlabeled data as having a special simple label and combine it with labeled data in the training process. Their method includes proofs that it can reliably learn the true underlying data patterns and that it performs better or at least as well as using only labeled data. Experiments on different types of data show that LACD is more efficient and generates better results than standard approaches.
Authors
Jin Su, Yuan Gao, Yong Zhou, Jian Huang
Abstract
Conditional diffusion models have become a powerful and flexible framework for learning complex conditional distributions from labeled data. In practice, however, acquiring high-quality labels is costly and time-consuming, leaving large volumes of unlabeled data unused. To address this, we introduce label-augmented conditional diffusion (LACD), a simple and effective approach that incorporates unlabeled examples by assigning them a designated trivial label and performing joint denoising score matching over the augmented dataset. We provide sufficient conditions guaranteeing population-level identifiability of the target conditional distribution under this scheme. Moreover, we establish rigorous statistical guarantees: when sufficiently many unlabeled samples are available, the sampling distribution produced by LACD converges strictly faster than the purely supervised estimator in total variation distance, and at least as fast in Wasserstein-1 distance. Extensive experiments on synthetic, image, and tabular benchmarks corroborate our theory and show substantial gains in sample efficiency and generative performance compared with the purely supervised estimator.