A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems
Machine LearningArtificial Intelligence
Summary
The authors took a complex model called DynaMix that predicts how systems change over time and simplified it to a very basic version named DynaBase with only two parameters. Despite being much simpler, DynaBase can still predict different types of system behaviors well without extra training. This simplicity makes it easier to understand how the model works and optimize it mathematically. Their analysis also connects different prediction methods into a single framework, explaining why some work better for short-term forecasts and others for understanding system dynamics over time.
Authors
Christoph Jürgen Hemmer, Florian Plaswig, Daniel Durstewitz
Abstract
Recent foundation models (FMs) for zero-shot reconstruction of dynamical systems (DS) achieve strong out-of-domain generalization but provide little insight into the mechanisms that underlie their forecasts. Such an understanding could help to strip down overladen FM architectures to their bare essence and expose the minimal requirements for in-context learning in the DS domain. Toward this goal, here we iteratively reduce a recent powerful SOTA model for DS reconstruction, DynaMix (Hemmer & Durstewitz, 2025), to a minimal interpretable two-parameter form, which we call DynaBase. DynaBase produces forecasts through a linear blend of the current latent state and the nearest in-context neighbor and its temporal successor. Surprisingly, despite its extreme simplicity, DynaBase produces highly competitive zero-shot DS reconstructions across chaotic and cyclic systems, with a negligible parameter load, many orders of magnitude below that of other FMs. Even more, this extreme simplicity permits direct model optimization on DS reconstruction measures, as well as closed-form one-step analytical solutions on prediction MSE. Theoretical and empirical analysis of DynaBase further leads to a 1-parameter family of maps, with the context-parroting algorithm of (Zhang & Gilpin, 2026) recovered at one end, and chaotic (divergent but bounded) behavior at the other. We further show how different training strategies lead to models either optimal for short-term prediction or for DS reconstruction. Thus, DynaBase not only exposes the minimal mechanisms required for producing zero-shot DS reconstruction, but also reconciles within an accessible mathematical frame divergent observations in the literature.