Gumbel straight flow improves fast language generation from models
Gumbel Straight Flow: Distilling Autoregressive Models into One-step Flow Maps
Machine LearningArtificial Intelligence
Summary
Generating sentences one word at a time can be slow and complex. The authors present a new method that turns a slow step-by-step language model into a faster, smooth process that produces sentences in just a few steps. They use a mathematical way to connect random noise to words so the process is clear and non-overlapping. This approach matches or beats current fast-generation methods on several language tasks.
What this means in practice
- •For natural language application developers: Generate text outputs quickly with fewer steps without losing quality using distilled autoregressive models.
- •For speech recognition software teams: Speed up transcription by employing one-step flow maps that maintain accuracy from complex language models.
Authors
Yeongmin Kim, Arnaud Doucet, Andrew Campbell, Valentin De Bortoli, Thomas Mensink, David Ruhe
Abstract
We present Gumbel Straight Flow (GSF), a continuous flow map language model that leverages the noise-data coupling of a pretrained autoregressive language (AR) model. We theoretically demonstrate that the coupling between Gumbel noise and one-hot token sequences induced by an autoregressive model yields non-intersecting linear paths connecting the noise to the sequence representations. To further enhance high-quality few-step path sampling, we use a flow map semigroup objective where the tangent (velocity) condition is guided directly by the AR teacher. Across various benchmarks, including pretraining and downstream tasks, GSF can outperform current few-step language generation baselines.