Mine odyssey benchmarks spatial agent intelligence in minecraft worlds
Mine Odyssey: Benchmarking Spatial Agentic Intelligence in the Wild
Artificial Intelligence
Summary
Helping agents navigate and act in real-world places is hard because they must explore, learn, and adapt in spaces they don't know. The authors created Mine Odyssey, a new set of 180 tasks in Minecraft versions of famous and varied real-world places. These tasks test how well AI agents can follow instructions, find paths, open doors, use stairs, and fix navigation mistakes. They found that current top models still make many errors, showing there is plenty of room to improve spatial intelligence in agents.
What this means in practice
- •For robotics engineers: Evaluate and improve robot navigation and interaction skills in complex environments modeled on real locations.
- •For video game developers: Create AI agents that navigate and respond to realistic spatial challenges within large and diverse virtual worlds.
Authors
Yuxuan Cao, Junlong Li, Hao Li, Junxian He
Abstract
Advances in foundation models are driving efforts to introduce agents to assist people in the physical world. Such agents require agentic spatial intelligence: exploring unfamiliar environments, updating spatial understanding through interaction, and adapting actions based on feedback to sustain progress toward a sequence of goals. Existing benchmarks cover only a limited range of spatial layouts, scales, and traversal requirements. We introduce Mine Odyssey, a benchmark for evaluating agentic spatial intelligence using Minecraft reconstructions of real-world locations. It comprises 180 tasks covering 30 such locations across 20 countries and regions on five continents, including 20 outdoor and 10 indoor settings. These settings span diverse spatial scales, layouts, terrains, and connectivity patterns, from Midtown Manhattan and rural Entrup to Santa Lucía Hill and Buckingham Palace. We select meaningful waypoints, such as landmarks, buildings, and rooms, and manually verify their accessibility. Each task provides a natural-language instruction specifying which waypoints to visit and in what order. Completing these tasks requires agents to find accessible routes and entrances, open doors, and move between levels using stairs and ladders, while monitoring their progress and recovering from navigation errors. Across eight evaluated state-of-the-art models, GPT-6 Astra achieves the highest success rate of 85.6%. However, the second-best model, Claude Opus 5.5, completes 73.9% of tasks, while the strongest evaluated open-weight model, DeepSeek-V4.1-Flash, reaches 23.9%, highlighting substantial room for improvement in the agentic spatial intelligence of current models. Comprehensive analyses and ablation studies on Mine Odyssey reveal current models' limitations and provide insights for advancing agentic spatial intelligence.