Method to measure energy water and carbon impact of AI tokens
Greenpixie's AI Token Methodology: Assessing the Energy, Water and $\mathrm{CO_2\text{-}eq}$ Impact of AI Tokens for Open and Closed Weight Models
Computers and SocietyMachine Learning
Summary
Running large language AI models uses energy and resources like water and emits greenhouse gases. The authors developed a way to estimate how much energy, water, and carbon emissions each AI token—small pieces of text generated or processed—costs. They measured energy use on GPUs and other server parts during AI use and used statistics to predict these costs even for AI models with unknown details. This helps companies understand the environmental impact of AI and find ways to reduce it.
What this means in practice
- •For cloud infrastructure teams: Quantify and reduce the environmental costs of running AI models in cloud data centers per token processed.
- •For ai platform operators: Estimate and report AI usage emissions and water consumption to customers for transparent sustainability metrics.
Authors
Joshua Horswill, Ross Hunter, Matt Clifford, James Hall
Abstract
We describe a methodology for estimating the per-token energy cost of cloud-hosted large language model (LLM) inference, separating between input (prefill) and output (decode) tokens. Graphics processing unit (GPU) energy usage is measured during inference benchmarking with open-weights models on a wide range of text-based tasks. The remaining server energy contribution from non-GPU hardware is estimated from the inference wall time. Bayesian linear regression is used to model the relationship between energy per token and LLM size, request traffic, and hardware deployment configuration. Proprietary frontier LLMs of unknown size and deployment are binned into size buckets based on naming conventions and performance priors, and the space of possible LLM configurations is sampled with Monte-Carlo methods to give a representative average energy per token and uncertainty. We also describe how these energy measurements can be used to estimate the carbon-dioxide equivalent ($\mathrm{CO_2\text{-}eq}$) emissions, both usage and embodied, and water consumed per token of AI inference. This methodology provides actionable data that enables reductions in cost, electricity usage, $\mathrm{CO_2\text{-}eq}$ emitted and water consumed in cloud and Software as a Service (SaaS).