Toward an Energy-Optimized Operation of Data Centers Located in Wind Farms Using Reinforcement Learning
2026-06-29 • Machine Learning
Machine Learning
AI summaryⓘ
The authors explore using Reinforcement Learning (RL) to manage shifting computer workloads in data centers powered by wind turbines. They created a simulation that models wind and electricity prices to test how well RL can make decisions without knowing the full day ahead. They found that basic RL struggles to use free wind energy effectively early on and tested two methods—Imitation Learning and Reward Shaping—to improve performance. While the RL methods improved, there is still a gap compared to an ideal optimizer that plans with full future knowledge. Their work sets a foundation for studying more complex systems with multiple data centers and continuous decisions.
Reinforcement LearningHigh-Performance ComputingWind EnergyWorkload ShiftingImitation LearningReward ShapingProximal Policy OptimizationSoft Actor-CriticOnline ControlSimulation Framework
Authors
Jan Stenner, Alexander Kilian, Sebastian Peitz, Hermann de Meer
Abstract
This paper studies Reinforcement Learning as an online controller for curtailment-aware workload shifting in wind-turbine-integrated high-performance computing (HPC) data centers. We introduce a reproducible fixed-day simulation framework with synthetic wind and price signals and delayed completion feedback, designed to be extensible toward more complex scenarios. As a controlled benchmarking basis, we then focus on the minimal case with one wind turbine and one co-located data center. In this setting, pure Reinforcement Learning exhibits a pronounced credit-assignment problem and tends to underuse free wind energy early in the day. We therefore evaluate two complementary countermeasures: optimization-based Imitation Learning and potential-based Reward Shaping. Across multi-seed training and a 200-day test set, Proximal Policy Optimization (PPO) and a Soft Actor-Critic (SAC) variant with an additional on-policy update routine achieve strong empirical performance among learned policies, and both Imitation Learning and Reward Shaping provide improvements in relevant configurations. A performance gap to the optimizer remains, which is expected: the optimizer plans offline with full-day foresight, whereas Reinforcement Learning must decide online from current observations without future realizations. The benchmark and ablation results provide a transparent basis for extending the approach toward richer multi-site and continuous-time scenarios.