Self-training generative models improved using geometric output changes
Improving Generative Model Self-Training with Geometrically Modified Outputs
Machine LearningArtificial Intelligence
Summary
When machines learn to create things like images or text, they often improve themselves by looking at their own work. But this can sometimes cause the machine to get worse instead of better. The authors found a way to change the machine’s outputs in a special geometric way that makes the problems more obvious, so the machine can use these clues to get better over time. This method helps self-learning systems produce higher-quality results consistently.
What this means in practice
- •For machine learning engineers: Improve generative models by enhancing self-training signals to reduce quality degradation and boost output variety.
- •For ai product developers: Develop more reliable generative AI features by incorporating geometric modifications during model finetuning to enhance long-term performance.
Authors
Patrick Batsell, Thomas Walker, Richard Baraniuk
Abstract
Self-training generative models - the continued improvement of a model using its own outputs - is becoming increasingly important as high-quality training data becomes scarce. However, naively finetuning on model-generated samples leads to degradation through model collapse and the model autophagy disorder. Negative-guidance self-training methods turn this degradation into a useful signal, using a model finetuned on its own outputs to guide the original model toward improved generation. Existing methods, however, take the negative signal in standard model outputs as given. We instead ask whether this signal can be explicitly strengthened. We introduce Geometrically Modified Outputs (GMOs), which reweight the singular values of the generator's input-output Jacobian to increase the influence of its leading singular directions. This geometric modification amplifies the mode-seeking behavior and distortions of standard outputs, providing a stronger and more targeted negative signal for self-training. Across a range of one-step generative models, GMOs consistently improve the performance of negative-guidance methods, including Neon and SIMS, compared with using standard model outputs.