Sense method improves world model planning for navigation and control
The Sense Paradigm for World Models
Machine Learning
Summary
Predicting future states in complex environments using computer models can be tricky because models can produce meaningless outputs when they simplify too much. The authors introduce Sense, a new way to measure changes between model predictions that prevents this problem without extra tricks. Sense helps models tell when actions can be reversed or not, making planning more reliable. Their tests show this method improves navigation success and detects important transitions better than previous methods.
What this means in practice
- •For robot navigation teams: Enable robots to plan routes more reliably by detecting irreversible transitions accurately in visual environments.
- •For game ai developers: Improve AI agents' ability to plan long-term actions in pixel-based game environments using the Sense distance measure.
Authors
Haidong Hu
Abstract
Latent world models predict the future in an embedding space rather than in pixels. An encoder trained only to predict collapses and maps every observation to the same point, so existing models add a mechanism against collapse, such as a target network, a variance penalty or a distribution penalty. These mechanisms keep the embedding from collapsing, but they leave its distances without meaning, and the model cannot tell a reversible transition from an irreversible one. We propose Sense, a single cross-entropy loss over the directed distance $d(x\to y)=\sum_i \mathrm{ReLU}(y_i-x_i)$ that replaces these mechanisms. The loss avoids collapse on its own, and since the encoder no longer needs the predictor, the predictor is fitted afterwards to fixed embeddings by plain regression. On four pixel environments, the return distance learned by Sense detects irreversible transitions with an AUC between 0.95 and 1.00, where LeWorldModel stays between 0.39 and 0.59. On TwoRoom, the navigation benchmark of LeWorldModel, planning with Sense reaches the goal in 100% of episodes against 91% for LeWorldModel. As the goal moves farther away, the planning failure rate of every baseline grows, while that of Sense stays near zero in Cup, Cliff and Sokoban and is the lowest of all methods in Pendulum.