Stabilizing unknown unstable continuous-time linear systems from data
On Stochastic Continuous-Time Linear System Identification for Adaptive Stabilization
Machine Learning
Summary
Stabilizing systems that change continuously over time and have randomness is a challenging problem. The authors study how to learn control settings to keep such systems stable by using observed data, even if the system is unstable initially. They found that longer observation times greatly improve stabilization, and that having a few long data runs is as effective as many short ones. Also, perfectly identifying the exact system details isn't necessary to stabilize it successfully.
What this means in practice
- •For control engineers: Design stabilizing controllers for industrial systems with unknown or unstable continuous dynamics using data-driven methods based on trajectory observations.
- •For robotics developers: Improve adaptive control in robots by stabilizing uncertain continuous-time linear models without needing full precise system identification.
Authors
Reza Sadeghi Hafshejani, Mohamad Kazem Shirani Faradonbeh
Abstract
Learning to stabilize an unknown continuous-time stochastic linear system is a central problem in control theory and reinforcement learning. This paper numerically studies data-driven identification of such systems with possibly unstable dynamics for the purpose of stabilization. We present a method that learns a stabilizing gain from observed trajectories, and examine how the failure probability, namely the probability that the learned gain does not stabilize the system, depends on the amount of data, dimension, noise intensity, excitation, and controllability. Numerical experiments on an unstable batch-reactor benchmark show that the squared identification error decays inversely with the aggregate observation time, that is, the combined length of all trajectories, while the failure probability decays exponentially in the same quantity. Hence, despite instability, a single long trajectory can achieve similar stabilization performance to that of many short trajectories of the same total length. We also observe \emph{signal saturation}: stronger excitation improves stabilization only logarithmically with the excitation strength. Finally, we show experimentally that accurate identification of the full dynamics is \emph{not} necessary for successful stabilization.