Breaking Block Boundaries: Anchor-based History-stable Decoding for Diffusion Large Language Models
2026-04-10 • Computation and Language
Computation and Language
AI summaryⓘ
The authors studied a new way to make large language models (called diffusion models) faster and better at generating text. They found that a common method, Semi-autoregressive decoding, slows down because it waits too long on some tokens. To fix this, they created a new method called Anchor-based History-stable Decoding (AHD) that watches when tokens become stable and then speeds up the process. Their tests show AHD makes decoding faster and improves performance without retraining the model.
Diffusion Large Language ModelsSemi-autoregressive decodingToken stabilityDynamic decodingAnchor-based decodingInference efficiencyCross-block decodingLanguage benchmarksPerformance improvementDecoding acceleration
Authors
Shun Zou, Yong Wang, Zehui Chen, Lin Chen, Chongyang Tao, Feng Zhao, Xiangxiang Chu
Abstract
Diffusion Large Language Models (dLLMs) have recently become a promising alternative to autoregressive large language models (ARMs). Semi-autoregressive (Semi-AR) decoding is widely employed in base dLLMs and advanced decoding strategies due to its superior performance. However, our observations reveal that Semi-AR decoding suffers from inherent block constraints, which cause the decoding of many cross-block stable tokens to be unnecessarily delayed. To address this challenge, we systematically investigate the identification of stable tokens and present three key findings: (1) naive lookahead decoding is unreliable, (2) token stability closely correlates with convergence trend, and (3) historical information is isolated. Building on these insights, we propose Anchor-based History-stable Decoding (AHD), a training-free, plug-and-play dynamic decoding strategy. Specifically, AHD monitors the stability trend of tokens in real time through dynamic anchors. Once a token reaches stability, it initiates early cross-block decoding to enhance efficiency and performance. Extensive experiments across language, vision-language, and audio-language domains demonstrate that AHD simultaneously improves both performance and inference efficiency. Notably, AHD effectively reverses the performance degradation typically observed in existing advanced decoding acceleration strategies. For instance, on the BBH benchmark, our approach reduces decoding steps by 80% while improving performance by 3.67%.