Summary
Controlling heating and cooling in buildings involves balancing energy use, comfort, and equipment health. This paper looks at traditional control methods like PID controllers used in thermostats and compares them to newer machine learning methods called reinforcement learning. The authors classify how reinforcement learning can be combined with or replace parts of existing control systems. They show that each approach affects how control signals are set and what information and modeling are needed. The paper suggests these methods are not competing but complementary ways to manage building climate systems.
What this means in practice
- •For building automation engineers: Use the taxonomy to select and integrate control methods adapting existing HVAC control systems with reinforcement learning components for better energy and comfort trade-offs.
- •For industrial control system designers: Incorporate hybrid control layers combining PID and reinforcement learning with model predictive control to handle complex nonlinear HVAC dynamics and constraints.
A survey. It maps existing work.
Authors
Erfan Haghighat Damavandi, Davide Papurello, Mahdi Alibeigi, Armin Keshavarz
Abstract
This paper reviews control architectures for building heating, ventilation and air-conditioning (HVAC) systems, addressing the tension among energy demand, occupant comfort and equipment operability, and situating proportional-integral-derivative (PID) regulation within a layered control stack that also includes rule-based and model-based supervisory control and data-driven reinforcement learning (RL). It examines the thermostat-PID layer from two-position and scheduled setpoint control through proportional, PI and PID regulation and their tuning, then formulates RL for building control as a Markov decision process. Its central contribution is a taxonomy of where RL meets PID: RL as a gain tuner for an existing loop, as a direct replacement for the regulatory controller, as a hybrid with model predictive control, and as supervisory setpoint and schedule generation. Across these classes, the review establishes that the physical loop remains invariant while the intervention point changes, and that the classes differ in their requirements for actuation authority, state information, modelling, commissioning and constraint handling. It further establishes that PID is inexpensive and well codified but limited by nonlinearity, time variation, discrete mode switching, loop interaction and slow plant dynamics, while RL's admissible position must be argued rather than assumed. The paper concludes that these architectures are not competing philosophies of control but nested points of attachment on one HVAC plant, with hybrid implementations possible, and that deployment trade-offs must be assessed according to the specific control interface and scaffolding required.