World-state reasoning improves robotic search in city environments
Structured World-State Reasoning for Agentic Robotic Search
Robotics
Summary
Robots searching large areas face challenges because they get information from many sources that can be unclear or incomplete, like maps, words, and their own observations. The authors created WORLDS, a system that keeps a map updated as the robot explores and uses multiple reasoning processes to make better decisions about where to look next. WORLDS was tested on thousands of city navigation tasks and showed much higher success rates than previous methods. It can also operate on a flying robot, successfully identifying specific targets even when they are not on the map.
What this means in practice
- •For urban search and rescue teams: Use WORLDS to guide robots in complex city environments for locating targets in incomplete maps and unclear conditions.
- •For drone operators: Deploy quadrotors equipped with WORLDS for autonomous search missions that identify and verify targets from onboard sensors.
Authors
Finley R. Holt, Luis A. Pabon, John Irvin Alora, Jonas Frey, Marco Pavone
Abstract
Long-horizon robotic search must resolve natural language against heterogeneous, incomplete, and often ambiguous evidence: textual information, prior maps, and observations arriving over time. The core challenge is to contextualize these streams and decide where to gather evidence before selecting a target. We present WORLDS: World-state Observation and Reasoning for Language-guided Discovery and Search, a framework that grounds reasoning in a persistent graph initialized from geospatial priors and updated by perception. Parallel Reasoners maintain competing candidate interpretations and request evidence to distinguish between them. We collect and process the requested observations with a multimodal Examiner, after which a Judge selects a grounded target or requests another pass. WORLDS achieves 51.8% navigation success across all 5,311 CityNav test episodes, the highest reported success rate, exceeding the previous published best by 15.7 percentage points under an OSM-only, high-resolution orthographic protocol. On 1,000 shared episodes, it achieves 50.0% versus 27.9% for the strongest adapted baseline using the same model, prior, sensing stack, and movement budget. Observation-based verification by the Examiner contributes 5.9 points of this success, and at a reduced reasoning-effort setting WORLDS still exceeds the adapted GeoNav baseline by 18.8 points while generating fewer tokens. We also demonstrate WORLDS on a quadrotor, which flies the generated sensing waypoints and grounds three language targets, including a vehicle absent from the map, from its onboard imagery.