Language models generate text faster with new rectified flow approach
Rectified Language Flow
Artificial IntelligenceMachine Learning
Summary
Generating text with current fast methods often sacrifices quality compared to traditional step-by-step models. The authors introduce Rectified Language Flow (ReLF), which creates text by gradually shaping random noise into meaningful words using a new flow technique combined with smart training tricks. They show ReLF can generate long sentences much faster while keeping or improving quality over popular models. This approach could make AI language generation more efficient and practical for real-time use.
What this means in practice
- •For natural language processing engineers: Accelerate generation of long text sequences in applications needing both speed and quality by replacing autoregressive models with ReLF.
- •For real-time chatbot developers: Enable much faster text response generation in chatbots and virtual assistants while maintaining output coherence and diversity using this flow-based model.
Authors
Baiyu Su, Runlong Liao, Lizhang Chen, Qiang Liu
Abstract
Continuous-state flow and diffusion models offer a promising approach to fast, parallel language generation, but matching the generation quality of autoregressive models remains challenging. We present Rectified Language Flow (ReLF), a language model that generates text by transporting noise to token embeddings using rectified flow. ReLF combines several complementary design choices: fixed token embeddings constructed with Hellinger-SVD, self-conditioning guidance trained with cross-entropy, and regularized reflow for few-step generation. We further introduce singular logits, which combine an analytically computed term that drives commitment to a valid token embedding as noise vanishes with a bounded neural residual that learns contextual token preferences. With 64 function evaluations on OpenWebText, ReLF achieves lower generative perplexity than autoregressive baselines at matched diversity across all three model sizes tested, while also outperforming prior continuous flow methods in this setting. After two reflow rounds, ReLF still outperforms autoregressive baselines with 16 function evaluations while generating 1,024-token sequences 15-19 times faster at batch size 1.