Hybrid PID and deep reinforcement learning improves industrial control
Augmenting PID Control with Deep Reinforcement Learning: A Hybrid Approach to the Industrial Benchmark
Machine Learning
Summary
Industrial machines and processes can be very complex, making traditional controllers struggle to keep them running smoothly. The authors combine classic PID controllers with deep reinforcement learning, which can find clever control strategies but can be unstable on its own. Their approach uses deep learning to find the best settings for part of the PID controller, then relies on the PID for steady and reliable control. This mix achieves the good performance of deep learning with the trustworthiness of classic control systems.
What this means in practice
- •For industrial control engineers: Use deep reinforcement learning to optimize PID controller parameters for complex industrial processes while keeping system reliability.
- •For automated process developers: Enhance control software by incorporating deep learning discovered settings into traditional controllers for better multi-input nonlinear system management.
Authors
Zhengyang, Gu, Joseph E. Hernandez, John Burtenshaw, Sean Scott, Thomas Cook, Chris Couch
Abstract
As industrial processes grow in complexity, traditional Proportional-Integral-Derivative (PID) controllers are often insufficient for handling their non-linear, multi-input dynamics. We propose using advanced Deep Reinforcement Learning (DRL) to prove its advantages in these complex environments. To do this, we rely on the Industrial Benchmark (IB). The IB is a realistic simulation that tests DRL algorithms against the key challenges of industrial applications: high-dimensional state spaces, delayed effects, and conflicting multi-criterial objectives. This testbed highlights DRL's core trade-off: while its final policies can often be unstable, its unique strength is the ability to autonomously discover optimal, non-obvious policies in multi-dimensional spaces where simple controllers fail. In this paper, we propose a novel hybrid PID-RL controller that leverages DRL's discovery capability while ensuring Reliability. After developing a multi-objective reward function to make DRL viable, we use a twin-delayed deep deterministic (TD3) agent as a discovery tool to find the optimal, non-obvious settings for the IB's 'Gain' and 'Shift' parameters. By feeding these discovered parameters to a simple, tuned PID controller, our hybrid model successfully combines all three characteristics: it achieves the optimal Performance and Efficiency of the best DRL agent with the Reliability of a classical controller. This work demonstrates a practical methodology for using DRL to augment, rather than replace, trusted industrial control systems.