Koopman operator methods improve chaotic system prediction accuracy
Understanding Latent-Dimension Scaling in Dynamical-System Learning through Spectral Reliability
Machine Learning
Summary
Predicting how complicated systems change over time is hard, especially when the systems behave chaotically. The authors studied ways to represent these systems in simpler, smaller forms that still allow good predictions. They found that models focusing on the reliability of key mathematical features called eigenvalues made better long-term predictions than ones focusing just on one-step errors. This suggests paying attention to these features helps better understand and improve how predictions improve when using bigger, more detailed hidden spaces.
What this means in practice
- •For control engineers: Build more accurate models of chaotic systems to improve long-term control by selecting latent dimensions with reliable system eigenvalues.
- •For financial modelers: Enhance predictions of complex financial time series by using spectral reliability to choose effective latent space sizes in dynamic models.
Authors
Itsushi Sakata, Yuta Miyauchi, Yoshinobu Kawahara
Abstract
In deep learning, approximation theory motivates increasing representation size. We ask whether this benefit extends to dynamics learning through autoregressive prediction. We analyze the learned time evolution through the eigenstructure of Koopman operators, using relative residuals to detect spurious eigenpairs arising even as one-step error falls. For bounded Koopman operators, we show that minimal residuals over learned dictionary spaces converge pointwise to their full-space counterparts as these spaces approximate the observable space in $L^2$. Our hypothesis is that Koopman spectral reliability helps explain how consistently rollout error decreases with increasing dimension. We compare two models of a shared Koopman autoencoder trained alternately for reconstruction and latent evolution, using latent-prediction loss (one-step prediction errors in latent coordinates) or spectral-residual loss (relative residuals of candidate eigenpairs). Across six chaotic systems, both models reduced median windowed rollout error from smallest to largest dimension. The spectral-residual model achieved lower medians than the latent-prediction model for all systems and dimensions, and its median fell by a larger factor in every system. Its median decreased monotonically with dimension in four systems, against one for latent prediction. Against four baseline families, its mean valid prediction times were nearly always longer. At the largest dimension under two-stage training, we compared eigenvalue positions with each learned dictionary's residual contours. Spectral-residual eigenvalues concentrated in low-residual regions, whereas latent-prediction eigenvalues also appeared in high-residual regions, consistent with the hypothesis.