One-step prediction methods fall short for long-term world modeling
One-Step Next-Latent Prediction Is Not a World Model
Computer Vision and Pattern RecognitionMachine Learning
Summary
Trying to predict the next step in a sequence by looking only at the current state can’t fully capture how things will change over time, especially for complex systems. The authors show that one-step predictions identify an average next state but miss critical details needed to accurately forecast far into the future. They demonstrate this through both mathematical reasoning and examples, like a simple time series and a hidden rotation, where multi-step predictions or using more past information lead to better accuracy.
What this means in practice
- •For machine learning engineers: Construct more accurate sequence models by incorporating multiple past steps instead of relying on one-step latent predictions alone.
- •For signal processing teams: Improve long-term forecasting in time series by using memory windows beyond single-step predictions to reduce error growth over time.
Authors
Shitong Wang, Zhongang Cai, Yuzhou Hong
Abstract
Next-latent prediction fits a map from the current embedding to the next one. LeNEPA carries this objective to time series, replacing the stop-gradient of next-embedding prediction with the isotropy penalty of LeJEPA. A world model is a transition kernel that can be rolled out. The one-step regression identifies a conditional mean, and a mean is a kernel only in special cases. For a linear-Gaussian Markov latent, the mean transition and the innovation covariance are fixed by the one-step problem, and the open-loop squared error at horizon $K$ equals the trace of the sum of the pushed-forward innovation covariances. That error grows with $K$ after the one-step fit is exact. If the conditional mean is nonlinear, composing it is not the multi-step conditional mean. If the observation is a non-injective function of a Markov state, a memoryless one-step map does not determine future observations, while a short window can. An isotropy penalty is a function of the embedding marginal, so its partial derivative in the transition weights is zero. On a scalar autoregression with coefficient $0.9$, the one-step mean squared error is $0.998$ and the $16$-step open-loop error is $5.10$. On a hidden rotation, an eight-step window reaches $16$-step error $0.056$, while the current scalar alone reaches $0.778$. Raising the isotropy weight from $0.1$ to $10$ leaves eight-step latent error inside $[0.78,0.85]$ on three seeds.