Semi-Supervised Conditional Generative Learning through Stochastic Interpolation and Sufficient Representations
Machine Learning
Summary
The authors address the challenge of generating data when labeled examples are limited but unlabeled data is plentiful. They propose a two-step method called RepG that first learns simple, low-dimensional patterns related to labels, then recreates high-dimensional data using these patterns. This approach allows the model to learn efficiently with fewer labels by focusing supervised learning on a smaller space while using unlabeled data to help reconstruction. Their theory shows that this method reduces errors and improves learning speed compared to traditional methods that work directly in high-dimensional space. Overall, the authors provide both theoretical and practical evidence that RepG helps overcome the difficulties of working with complex data and few labels.
Authors
Changyu Liu, Yuling Jiao, Jian Huang
Abstract
Conditional generative modeling remains a challenging problem in semi-supervised settings where labeled data is scarce but unlabeled samples are abundant. To effectively leverage structural information embedded within the unlabeled dataset and compensate for sparse conditioning signals, we propose a semi-supervised framework combining conditional stochastic interpolation with low-dimensional latent representations. RepG decomposes generation into two stages: label-dependent latent sampling and high-dimensional reconstruction. This isolates the supervised learning of conditional dependencies to a low-dimensional space, requiring few labels while utilizing the abundant unlabeled data purely for reconstruction. Theoretically, we establish an error decomposition showing that the Kullback-Leibler divergence of RepG comprises stage-wise estimation errors and a structural bias quantified by conditional mutual information. For deep neural network estimators, we derive non-asymptotic convergence rates proving that RepG significantly improves sample complexity. By confining the supervised estimation burden to the low intrinsic dimension of the latent representation, RepG achieves a strictly faster convergence rate. Complemented by a minimax lower bound, our theoretical results demonstrate that this method effectively mitigates the curse of dimensionality inherent in direct ambient-space generative modeling.