Generalization as a robust performance property of learning-enabled dynamical systems
2026-08-31 • Machine Learning
Machine Learning
AI summaryⓘ
The authors study how learning algorithms behave when given new data by treating changes in data like disturbances in a system. They model the impact of swapping a single data point as an external influence and analyze how the learning method responds using system theory tools. Their approach creates a way to measure and compare how well different learning methods generalize, including popular ones like gradient descent and momentum-based algorithms, and it also works for control systems driven by data. This provides a new perspective to guarantee that learned models will perform reliably on unseen data.
algorithmic stabilitygeneralizationdynamical systemsintegral quadratic constraintdissipativitygradient descentmomentum methodsNesterov accelerationdata-driven controlout-of-sample bounds
Authors
Filippo Fabiani
Abstract
By focusing on algorithmic stability as a means of establishing out-of-sample bounds, we provide a system-theoretic interpretation of generalization in learning-enabled dynamical systems arising in data-driven optimization and feedback control approximation. Given two neighboring datasets, we specifically model sample replacement as an exogenous disturbance acting on a sensitivity system, while the incremental behavior of the data-dependent operator is encoded through an integral quadratic constraint. By relying on dissipativity arguments, we establish a matrix inequality-based certificate and a uniform stability bound that separates the one-sample sensitivity of the learned operator, and an algorithm-dependent dynamical gain. The latter can then be optimized, offering a tractable tool for certifying and comparing generalization capabilities of learning dynamics. We show that our results recover classical ones for gradient descent, apply naturally to momentum-based methods such as heavy-ball and Nesterov acceleration, and extend to data-driven control.