Control unknown changing systems with low regret and fast detection
Logarithmic Regret via Passive Change Detection in Piecewise-Stationary Self-Tuning Regulation
Machine LearningArtificial Intelligence
Summary
This paper addresses how to control systems whose behavior changes unpredictably over time. The authors propose a method that quickly detects these changes without too many false alarms and adjusts the control accordingly. Their technique keeps errors growing slowly (logarithmically) over time, which means the system adapts nearly as well as if it knew the changes in advance. They do this by monitoring output energy and updating estimates during control, ensuring stable and informed adjustments.
What this means in practice
- •For control engineers: Manage variable industrial processes that change unpredictably by detecting changes quickly and adapting controllers with minimal performance loss.
- •For robotics teams: Maintain stable robot operation despite sudden changes in dynamics by using fast, reliable detection with guaranteed performance bounds.
Authors
A. Ch. Madhusudanarao, Rahul Singh
Abstract
We study minimum-variance control of an unknown autoregressive system with exogenous inputs and coefficients that change at unknown times. Under bounded independent disturbances, fixed detection gaps, stability and feasibility conditions, and sufficient time between changes, we prove \(O((C+1)\log((T+1)/δ))\) regret with probability at least \(1-δ\), where \(T\) is the horizon and \(C\) the number of changes. Unlike switching bandits, where unselected arms can change unobserved, admissible plant changes provide information during exploitation: the correct feasible controller leaves only the disturbance in the output, whereas a detectable change raises output energy under the old controller. PIECE-CD explores initially and after alarms, then uses gated recursive least squares for control. Its energy test compares windowed output power with a threshold above the noise floor; the extension to unstable controller mismatches also monitors the reference controller's input proposal. We control false alarms across the horizon and prove logarithmic detection delay. Inputs are clipped to prescribed bounds. Logarithmic regret also holds under an explicit condition ensuring that clipping becomes inactive after a finite burn-in. Under the stated feasibility conditions, the extended detector covers destabilizing changes with detectable excess energy over a fixed window.