Compact intervention structures persist in high-dimensional recurrent model dynamics
Compact but Moving: Intervention-Relevant Geometry in Recurrent World Models
Robotics
Summary
This paper explores how small but meaningful changes (interventions) in complex time-based computer models behave internally. The authors found that although models have many moving parts, the key corrections to their behavior can be represented in a compact, changing geometric pattern rather than a fixed low-dimensional space. This pattern helps predict how the model will respond to changes over time and is consistent across different types of recurrent neural networks. Their work shows that concise intervention information can move and adapt internally without losing its usefulness.
What this means in practice
- •For reinforcement learning engineers: Improve interpretability of recurrent models by tracking geometric corrections that predict model response to interventions during rollout.
- •For robotics control teams: Enhance control algorithms using compact, moving intervention representations to better anticipate consequences of action changes in recurrent models.
Authors
Yuming Chen, Yang Liu
Abstract
Learned world models may have compact interventions even when their recurrent state is high-dimensional, but it is unclear what happens to such a correction after it enters the model. We study this question in a controlled recurrent world model where prior work identified a checkpoint-specific rank-4 interface for one-shot counterfactual velocity interventions. The correction rapidly leaves this fixed entry subspace during autonomous rollout. Nevertheless, a low-rank image obtained by transporting the entry directions through the factual recurrent Jacobian chain continues to capture most of the nonlinear correction. Restarts using the tangent-predicted correction preserve substantial counterfactual future function. This transport/function pattern recurs across independently trained structured-GRU models and a parameter-matched LSTM initialized with a privileged compact correction. We further characterize a finite-horizon future-response operator over the full recurrent carrier. Patching shifts its leading future-sensitive directions toward the matched native-counterfactual organization, and the local operator accurately ranks finite perturbation effects over the registered direction panels at the patched and native-counterfactual basepoints. A separate full-amplitude assay finds substantial factual-endpoint tangent residuals and supports response reconfiguration in two of three checkpoints. Together, these results show that compact intervention structure can persist as a moving, state-dependent local geometry embedded in high-dimensional recurrent dynamics, without implying a fixed or dynamically closed low-dimensional state.