Offline rl selects faulted lines accurately in simulated power grids
Offline Reinforcement Learning for Distribution-Grid Protection
Machine Learning
Summary
Protecting electrical lines in power distribution networks is tricky because conditions change a lot. The authors tested a method called offline reinforcement learning to identify and isolate faulty lines using simulated data. Their system analyzes electrical signals to decide which line to trip, achieving high accuracy in test scenarios. However, the study was done under limited conditions and more work is needed to confirm how well this would work in the real world.
What this means in practice
- •For power grid operators: Improve automated fault detection and isolation decisions in distribution networks using offline trained reinforcement learning models.
- •For electrical protection engineers: Benchmark advanced machine learning techniques to complement traditional protective relays with data-driven line fault detection.
- •For industrial equipment suppliers: Develop smarter relay devices integrating offline reinforcement learning for adaptive fault line selection in evolving distribution grids.$Commercial implications: This paper enables machine learning-based relay protection products that adapt to distributed energy changes for utilities.
Tested on simulated data.
Authors
Julian Oelhaf, Alexander Luce, Christian Bergler, Andreas Maier, Siming Bayer
Abstract
Data-driven protection may complement conventional relays in distribution grids whose operating conditions vary with distributed generation, switching events, and changing short-circuit levels. We study line-selective tripping from static trajectories of a realistically simulated CIGRE medium-voltage network using offline reinforcement learning. A convolutional Q-network receives causal voltage-current phasor and apparent-impedance features, optionally together with raw waveforms, and is trained with conservative Q-learning (CQL). A controlled sensitivity study evaluates two observation windows, reward variants, and three CQL weights under a common split and training protocol; one exploratory post-hoc run additionally increases the discount factor from $γ$=0.95 to 0.99. On 225 held-out episodes, the best per-timestep result is obtained with combined input and CQL weight $α$=0.9, reaching precision 0.9993, recall 0.9496, and F1-score 0.9738. Because dense per-timestep scores do not encode the terminal semantics of relay operation, we also evaluate the first non-wait action in each episode. The default combined-input agent selects the correct line-trip action first in 98.13% of 214 fault episodes, but trips in 72.73% of the 11 non-fault episodes. In the post-hoc run, the corresponding rates are 98.60% and 54.55%, respectively. The results show that dense predictive performance and terminal protection behavior can lead to different model rankings. Offline CQL therefore demonstrates strong faulted-line selection on the simulated fault episodes, while the static trajectories, small non-fault set, and single-seed post-hoc design preclude conclusions about practical relay security or deployment readiness.