Kernel method improves learning for nonlinear system forecasts
Bilevel optimization for data-driven learning of Koopman embeddings using kernel-based autoencoders
Machine Learning
Summary
It can be hard to make predictions about complex systems that change in nonlinear ways, like the weather or ocean patterns. The authors created a new way to learn simpler linear models from data using kernel methods, which can be easier to interpret than neural networks. Their approach learns the important patterns directly and works well compared to existing methods, even on large datasets like video or sea temperature data. This makes it easier to build accurate and scalable models to understand and forecast nonlinear systems.
What this means in practice
- •For climate modelers: Create more accurate sea-surface-temperature forecasts by learning interpretable linear representations from observational data.
- •For video analytics teams: Develop scalable models that identify dynamical patterns directly from video data for improved nonlinear system monitoring.
Authors
Joel-Pascal Ntwali N'konzi, Feliks Nüske, Stefan Klus
Abstract
Koopman operator theory provides a linear framework for analyzing nonlinear dynamical systems and has become a major tool for data-driven modeling. A central challenge, however, is that finite-dimensional approximations computed by methods such as extended dynamic mode decomposition (EDMD) require the dictionary to be specified a priori. Recent machine-learning approaches address this limitation by learning the dictionary from data, predominantly using artificial neural network (ANN) autoencoder architectures. Although kernel methods offer an alternative with greater interpretability and tractability for theoretical analysis, they have received little attention in this setting. We introduce extended dynamic mode decomposition with kernel-based dictionary learning (EDMD-kDL), a kernel-based method for learning finite-dimensional Koopman embeddings directly from data. The method combines ideas from collocation methods and bilevel optimization to simultaneously learn a kernel dictionary and the corresponding Koopman approximation. We evaluate EDMD-kDL against state-of-the-art ANN-based approaches on a range of numerical experiments, including global sea-surface-temperature forecasting and learning directly from video data. Across all tested settings, EDMD-kDL achieves performance comparable to or better than the ANN-based methods. Moreover, in contrast to standard kernel methods, the proposed approach is scalable to large datasets by design since the size of the required kernel matrices depends on the number of collocation points rather than the size of the training dataset.