Transformer shows detailed internal map despite navigation errors
World Modeling in Transformers
Artificial IntelligenceComputation and Language
Summary
Sometimes AI models that seem lost are actually keeping track of their surroundings in clever ways. The authors studied a transformer navigating Manhattan and found it does have an internal map, including streets and intersections. Its mistakes come from confusing similar spots because it merges details together too much. They suggest focusing on how a model represents and uses its environment instead of just asking if it has a world model.
What this means in practice
- •For robotics developers: Improve robot navigation by analyzing how transformers internally represent spatial environments to handle localization errors.
- •For autonomous vehicle teams: Develop better situational awareness systems by understanding how goal direction and position tracking emerge in AI navigation models.
Authors
Pierre Beckmann, Matthieu Queloz, Andre Freitas
Abstract
Behavioral failures can make a transformer appear to lack a world model even when it has learned faithful representations of its environment. We demonstrate this in TaxiGPT, a transformer trained on random walks through Manhattan whose failures have been interpreted as evidence of an incoherent internal map. Through mechanistic analysis and causal interventions, we show that the model represents intersections and streets, tracks its position, and uses a goal compass to navigate. We trace its failures to interference between superposed intersection features, which disrupts localization within the internal map. Affordance packing, which groups representations of intersections with the same legal moves, helps limit the consequences of these errors. Finally, we propose mechanistic indicators that we use to compare models and show that world-modeling capacities emerge at different stages of training. Our findings motivate a shift from asking whether a model has a world model to mechanistically studying its world modeling: the interacting capacities through which it represents its environment and uses those representations to guide behavior.