Time-resolved GPU power data reveals AI compute electrical dynamics
LLM-PowerTrace: A Dataset of Time-Resolved GPU Power Traces for AI Compute
Artificial IntelligenceOperating Systems
Summary
AI computers, especially those running big language models, use electricity in ways that can change very quickly and affect power systems. Existing data only show total energy use, missing how power spikes or fluctuates over short times. The authors created LLM-PowerTrace, a publicly available dataset that records detailed power use every 0.1 seconds on GPUs during AI training and running tasks. Their data helps others understand and predict the fast electrical changes caused by AI workloads on different GPUs and setups.
What this means in practice
- •For data center operators: Manage and forecast AI workload power fluctuations to ensure reliable power supply and infrastructure stability.
- •For cloud service managers: Optimize GPU resource scheduling by understanding fine-grained power usage under varying AI model settings and workloads.
Authors
Jiansheng Wang, Jaskeerat Singh, Haochen Xu, Xinyi Li, Mingyuan Yan, Shaohui Liu, Yuzhuo Li, Yize Chen, Wenqi Cui
Abstract
AI compute is becoming large and fast-changing electrical loads. Its impacts on data center infrastructure and power grids depend not only on total energy consumption, but also on how their power draw moves over sub-second to minute timescales, including peaks, ramps, and synchronized fluctuations across accelerators. Existing energy benchmarks for large language models (LLMs) report aggregate quantities such as joules per token and rarely release the underlying time-series measurements, leaving these power dynamics poorly characterized and difficult to model. We introduce LLM-PowerTrace, an open dataset of time-resolved, per-GPU power traces for LLM fine-tuning and inference, sampled every 100\,ms on varying NVIDIA platforms paired with complete workload and system metadata. For fine-tuning, LLM-PowerTrace profiles open model families spanning a wide range of model sizes and fine-tuning methods, together with controlled sweeps over batch size, sequence length, precision, parallelism, and checkpointing. For inference, it covers variations in concurrency, GPU frequency, KV-cache and weight quantization, prompt and output lengths, and request-arrival patterns. LLM-PowerTrace profiles a set of open model families across a wide range of model sizes, and controlled sweeps over batch size, sequence length, precision, parallelism, and checkpointing for fine-tuning, and it sweeps concurrency, GPU frequency, KV cache and weight quantization, prompt and output lengths, and request arrival patterns for inference. Built upon power traces, we define power-grid-facing metrics as well as AI-compute power\&energy scaling law for extrapolating beyond the measured settings. Together, LLM-PowerTrace provides a dataset foundation for modeling, forecasting, and managing the fast-changing electrical behavior of AI compute.