Control of unknown systems with safe learning using Bayesian methods
A Bayesian Framework for Safe Optimal Control of Unknown Systems from Input-Output Data
Machine Learning
Summary
Controlling complex systems is hard when you don't know all the details or can't measure everything directly. The authors present a way to learn how a system behaves safely by using data of inputs and outputs without knowing its inner states. Their approach uses a Bayesian method to understand uncertainty in the system's behavior and ensures control actions keep the system within safe limits. They also show how the controller can improve over time by updating its knowledge as new data comes in.
What this means in practice
- •For control engineers: Design controllers that safely operate systems with unknown nonlinear dynamics using only input-output data and without full state measurement.
- •For automation operators: Implement adaptive controllers that improve safety and performance over time by progressively learning system behavior during operation.
Authors
Robert Lefringhausen, Annalena Daniels, Michael Fink, Sandra Hirche
Abstract
As control engineering is applied to increasingly complex systems, learning-based control, which uses empirical data to reduce the reliance on first-principles models, has emerged as a promising paradigm. However, such approaches must cope with nonlinear and stochastic dynamics, the absence of state measurements, and scarce data, while ensuring safe operation under constraints, which requires rigorous uncertainty quantification. Whereas existing methods treat these challenges in isolation, this article proposes a Bayesian framework that jointly addresses them. A prior in state-space form is updated from input-output data using particle Markov chain Monte Carlo methods, yielding a representation of joint uncertainty over dynamics and latent states. Building on this representation, a validation procedure is developed that provides probabilistic guarantees for constraint satisfaction under arbitrary causal control laws. For control synthesis, a formulation penalizing the conditional value-at-risk of the constraint function is proposed, together with an adaptive calibration mechanism that adjusts the performance-robustness trade-off to achieve a desired robustness level. The probability of successful certification is characterized theoretically. The framework is extended to a receding-horizon setting, where posterior updates from new measurements allow the controller to progressively reduce uncertainty and improve performance. The approach is demonstrated on the optimal control of a vertical farm.