DynaTE speeds up diffusion large language model decoding by skipping tokens
DynaTE: Accelerating Diffusion LLMs via Dynamic Token Execution
Hardware ArchitectureArtificial Intelligence
Summary
Generating text with large language models (LLMs) can be slow when tokens are processed one by one. Diffusion-based LLMs speed this up by refining many tokens in parallel, but hardware is often not optimized for this approach. The authors introduce DynaTE, a new system that dynamically skips unimportant tokens and adjusts computation based on changing token needs, making the whole process faster and more energy efficient. They show DynaTE achieves over twice the speed and energy efficiency compared to current specialized hardware.
What this means in practice
- •For llm hardware designers: Design accelerator chips that dynamically skip low-utility tokens to improve speed and energy efficiency in diffusion-based large language model decoding.
- •For edge device developers: Build energy-efficient AI inference solutions on limited hardware by integrating DynaTE’s streaming vocabulary engine and adaptive token execution.
Authors
Minghan Jiang, Jiayi Wang, Shuaiting Li, Haibin Shen, Kejie Huang
Abstract
Diffusion-based LLMs (dLLMs) have recently emerged as a promising alternative to autoregressive (AR) LLMs by enabling bidirectional parallel refinement, alleviating the sequential decoding bottleneck of AR generation. However, their parallel iterative refinement mismatches AR accelerators optimized for sequential decoding and their discrete token generation differs from DiT accelerators designed for continuous denoising. Recent dLLM accelerators have explored workload-specific optimizations to reduce vocabulary processing overhead and redundant computation across denoising iterations. However, these approaches retain all tokens in parallel execution, despite varying token refinement utility and execution requirements. This paper presents DynaTE, a hardware--software co-design architecture that dynamically adapts accelerator execution to evolving token states during dLLM decoding. DynaTE first enables adaptive token execution by skipping low-utility token computation, while a dimension-reconfigurable PE array maintains high utilization under varying active-token patterns. Second, DynaTE exploits dynamic token dependencies through FLDD to refine a small number of locally dependent tokens within the current iteration, reducing the overall number of denoising iterations, while a Merge--Split--Merge dataflow hides the resulting serial overhead. Third, a streaming vocabulary engine interleaves multiple token streams from the LM head to accommodate irregular output variations caused by selective token computation and uneven vocabulary-selection demands. Evaluated on two representative dLLMs, DynaTE achieves 2.05--2.78$\times$ speedup and 2.99--3.93$\times$ higher energy efficiency over state-of-the-art dLLM accelerators, while delivering 2.55$\times$ speedup and 6.07$\times$ higher energy efficiency over Jetson AGX Orin.