Learning Chaotic Dynamics through Second-Order Geometric Supervision
2026-06-01 • Machine Learning
Machine Learning
AI summaryⓘ
The authors study how to better learn chaotic systems from data by not only matching immediate predictions and slopes (first-order) but also how the system curves (second-order). They show that existing methods matching values and their tangent directions can still drift away from true behavior because they ignore curvature. To fix this without the large computational cost of directly computing curvature (the Hessian), they propose a technique called randomized Jacobian matching that indirectly enforces second-order consistency. Their experiments on classic chaotic models demonstrate improved long-term accuracy and stability using this approach compared to first-order methods.
chaotic dynamical systemsattractorJacobianHessianTaylor expansionLyapunov exponentLorenz systeminvariant measurerandomized Jacobian matchingvector field
Authors
Shinhoo Kang, Hai V. Nguyen, Tan Bui-Thanh
Abstract
Learning chaotic dynamical systems from data requires more than short-term predictive accuracy: the learned model must preserve the attractor geometry and its invariant statistics. Trajectory (zero-order) and Jacobian (first-order) matching supervise the values and tangent structure of the vector field, but neither constrains how the field bends away from its tangent plane. A model can thus match values and tangents at the supervised states yet curve differently from the truth, remaining locally accurate while drifting toward spurious attractors and distorting long-time statistics. We show that enforcing second-order consistency mitigates these failures, but forming the full Hessian is prohibitive in high dimensions. We propose model-constrained randomized Jacobian matching, which compares the Jacobians of the true and learned vector fields at randomly perturbed inputs. A Taylor expansion shows that the expected randomized Jacobian loss decomposes into the nominal Jacobian mismatch plus a Hessian mismatch scaled by the noise variance, implicitly enforcing second-order consistency at $\mathcal{O}(d^2)$ cost without forming the $\mathcal{O}(d^3)$ Hessian tensor. Using only Jacobian evaluations, the method scales to high dimensions where explicit Hessian matching does not. Numerical experiments confirm that second-order methods are robust. For Lorenz~63, first-order methods produce catastrophic Lyapunov-exponent outliers under minimal temporal supervision, which second-order methods eliminate while recovering the correct attractor. For coupled Lorenz~96, an out-of-distribution forcing sweep separates the methods: all agree up to $F=16$, but beyond $F=18$ only second-order methods preserve the invariant measure and Lyapunov spectrum. On both systems, randomized Jacobian matching performs comparably to explicit Hessian matching at much lower cost.