Accelerating Diffusion Language Models via Structured Suffix Modeling
2026-08-24 • Computation and Language
Computation and Language
AI summaryⓘ
The authors studied how to make Diffusion Language Models (DLMs) faster when they generate text by guessing many words at once but needing to check all the later words repeatedly. They found that treating different parts of the 'suffix' (the later words) differently—dividing it into local, middle, and tail regions with varying detail—helps speed up this process. They also improved speed by updating token information at each step rather than restarting it every time. Their method doesn’t require retraining the model and works well with other speed-up tricks, resulting in much faster text generation, especially for long sequences.
Diffusion Language Modelsparallel decodingsuffix windowdenoisingtoken representationsinference accelerationKV cachelong-sequence inferencestructured suffix modeling
Authors
Zifeng Cheng, Keda Li, Zhiwei Jiang, Cong Wang, Fei Shen, Qing Gu
Abstract
Diffusion Language Models (DLMs) exhibit strong parallel decoding capabilities by denoising multiple tokens in a single generation step. However, this parallelism comes with substantial computational overhead, as each step requires interactions with all suffix tokens. Existing methods typically reduce this cost by retaining only a local suffix window as a substitute for the full suffix. Despite their effectiveness, these methods overlook the structural heterogeneity across suffix regions and re-initialize suffix tokens with identical representations at each timestep. To this end, we propose a structured suffix modeling method for efficient DLM inference. Specifically, we divide the suffix into three regions, i.e., the local, middle, and tail regions, and retain different numbers of suffix tokens in each region according to their structural roles. Moreover, we incorporate the decoding results from the previous step into the suffix token representations at the current step, allowing them to carry evolving denoising information across generation steps. Notably, our method is training-free and orthogonal to several existing acceleration techniques, such as parallel decoding strategies and KV cache. Empirical results across multiple benchmarks on three DLMs demonstrate that our method can further accelerate DLM inference and improve performance in most cases. In particular, in long-sequence inference, our method achieves up to a \(72.81\times\) speedup when combined with other acceleration techniques. Our code is available at https://github.com/zifengcheng/SSM.