Reinforcement learning improves unit commitment and speeds up exact solvers
Reinforcement learning on the EPRI AI-ccelerating Unit Commitment benchmark: diagnosing over-commitment and guiding MILP with learned policies
Machine Learning
Summary
Deciding which power generators to switch on over time is a complex problem called unit commitment. The authors studied how well reinforcement learning (RL), a form of AI, could solve this problem compared to traditional optimization methods. They found that basic RL strategies tended to turn on too many generators, increasing costs, but by training the RL agents to mimic optimal solutions and fine-tuning them, they achieved near-optimal results. Moreover, using these trained RL policies to guide exact solvers made those solvers much faster without losing accuracy.
What this means in practice
- •For grid operators: Speed up exact unit commitment solvers by using RL-based policies to fix decisions, reducing computation time while maintaining near-optimal power dispatch schedules.
- •For energy system modelers: Use imitation-trained reinforcement learning policies to estimate unit commitment solutions faster and with cost accuracy close to traditional methods on large-scale benchmarks.
Authors
Maharani Devira Pramita, Aidan O'Sullivan
Abstract
Reinforcement learning (RL) is increasingly proposed as a fast solver for unit commitment (UC), but standalone agents remain far more expensive than mixed-integer linear programming (MILP). In the first study of RL on the 51-generator, 72-hour benchmark of the Electric Power Research Institute (EPRI) AI-ccelerating Unit Commitment competition, we trace this gap to the agents' own exploration, close most of it, and show that the resulting policy can accelerate an exact solver. In every configuration tested, Proximal Policy Optimisation (PPO) and an action-branching Deep Q-Network (DQN) trained from scratch committed about twice the units of a thermal-only MILP optimum, at 39-64% higher cost, whereas a merit-order rule came within 1.6-7% (10-24% when unserved energy is priced). When each unit's decision is sampled independently, spare units insure against random de-commitments. Consistent with this, injecting exploration noise during fine-tuning enlarged a lean policy's fleet, a reward without shaping terms did not remove the over-commitment, and under their training noise the agents outperformed the rule subjected to comparable noise. Imitating the MILP solutions and fine-tuning conservatively brought PPO within 2.6% of the optimum, and these policies had the lowest penalised cost of any evaluated (about 5%). Fixing the fine-tuned policy's confident decisions made the MILP 3.8-8.5 times faster within 0.10-0.25% of the best solution, with a smaller worst case than an imitation-only policy fixing as many variables (1.1% against 8.9%). Because dispatch method and benchmark scope alone shift the reported optimality ratio by up to 3.6% and 9%, we propose a reporting protocol.