Masked diffusion models improve parallel text generation accuracy and speed
Reliable Parallel Decoding in Masked Diffusion Language Models
Computation and LanguageArtificial Intelligence
Summary
Generating text by predicting many words at once can be tricky because some guesses depend on others. The authors show that being confident about a guess doesn’t always mean it’s reliable when made together with other guesses. They found a way to safely pick words to lock in by checking which ones stay consistent through different checks and how uncertain earlier guesses are. Their method speeds up text generation while keeping it accurate, especially for math and coding tasks with certain language models.
What this means in practice
- •For language model developers: Increase throughput and accuracy for tasks in mathematical reasoning and code generation using masked diffusion decoding techniques.
- •For software engineers: Implement faster parallel token prediction methods in applications relying on advanced language models to reduce latency during complex text generation.
Authors
Zhenghao He, Bohan Liu, Guangzhi Xiong, Aidong Zhang
Abstract
Masked diffusion language models (MDLMs) can generate text efficiently by predicting multiple masked tokens in parallel, but predictions from the same forward pass are not necessarily reliable when committed together. We study when parallel commitment is reliable. Our diagnostics show that confidence alone does not determine a reliable commitment order: confident predictions near the end of the sequence can fix an answer before its supporting computations are established, and downstream predictions become less reliable as the uncertainty of their upstream context grows. At the same time, a single forward pass can already resolve several masked tokens, and predictions that remain stable across the final layers are more likely to be correct. Based on these findings, we propose Reliable Parallel Decoding (RPD), a training-free method that selects candidates by layerwise prediction stability and final confidence, and commits them under a cumulative entropy budget over their preceding masked positions. RPD defers predictions with uncertain upstream context while committing the remaining candidates in parallel, without relying on a fixed block schedule. Across mathematical reasoning and code generation benchmarks on LLaDA and Dream, RPD achieves the highest decoding throughput among the evaluated methods while maintaining or improving accuracy.