Diffusion language models use half the energy of autoregressive models
Energy Consumption Analysis of Discrete Diffusion and Autoregressive Language Models
Machine Learning
Summary
Generating text with large language models uses a lot of energy, and this is a growing concern. The authors compared two types of models: autoregressive ones that create text one word at a time, and diffusion models that can generate many words at once. They found diffusion models use about half the energy per word when creating longer texts. This suggests diffusion models could help reduce energy costs in language processing tasks.
What this means in practice
- •For ai infrastructure teams: Optimize inference costs by choosing diffusion models to reduce energy use for long text generation workloads.
- •For data center operators: Plan energy budgets more accurately when hosting language model services by considering model type energy profiles.
Authors
Boris Ruf, Marcin Detyniecki
Abstract
As Large Language Models (LLMs) continue to scale, their energy consumption during inference has become a critical concern. While Autoregressive (AR) models are the industry standard, Diffusion Language Models (DLLMs) have emerged as a promising alternative capable of parallel token generation. In this paper, we perform a comparative energy benchmark of diffusion models (LLaDA-8B, Dream-7B) against AR counterparts (Llama-3.1-8B, Qwen-2.5-7B). We analyze the trade-offs between model architecture, KV-cache optimization, and Joules-per-token efficiency. Our results show that at longer sequences (200--600 tokens), diffusion models achieve 2.5--2.7 J/token, approximately 2$\times$ more efficient than AR models with KV-cache (5.3--5.7 J/token). KV-cache provides substantial benefits for AR models at longer sequences (50--70\% energy reduction).