Hierarchical Fast--Slow ReAct Agent for Zero-Shot Object-Goal Navigation
2026-08-10 • Robotics
Robotics
AI summaryⓘ
The authors study how a robot can find objects in a place it has never visited before without prior training for that specific environment. Instead of only using a vision-language value map that forgets past info quickly, they make the robot remember and organize what it has seen in a semantic memory map and tagged pictures. Their system uses a two-layer approach: a fast reactive controller and a slower, thoughtful reasoning layer that reviews the stored memory to make better decisions. This method improves success rates over previous zero-shot object navigation approaches and shows that deliberating with saved memories is more effective than blindly picking the best current option.
Zero-shot object-goal navigationVision-language modelValue mapCoordinate-anchored memorySemantic gridPose-tagged keyframesReactive controllerDeliberative layerHM3DMP3D
Authors
Zhaochen Lan, Zhi Yang, Yuxiang Fu, Mengxiang Lin
Abstract
Zero-shot object-goal navigation (ZSON) requires a robot to find a named object category in a building it has never entered. The prevailing approach scores frontiers with a vision--language \emph{value map}: every decision is another argmax over the map as it currently stands, and the evidence behind that score is discarded the moment it is taken. Systems that place a large vision--language model inside the perception--action loop typically query it on a fixed schedule from the current view alone; a room the robot walked through minutes earlier is never reconsidered, and a failed call has no defined fallback. We turn what the robot has already seen into the object of deliberation. Our hierarchical fast--slow agent leaves the value-map controller running at every step and writes a \emph{coordinate-anchored memory} as it moves: a semantic grid of room types and confirmed object instances, together with a bounded store of pose-tagged keyframes. A VLM screens each candidate detection before it is written. A deliberative layer reads this memory in a bounded reason--retrieve--act loop. It wakes on structural events the reactive layer computes, reasons first over text, and recalls a first-person view only for candidates that text alone cannot separate. Per-invocation and per-run caps bound its calls, a call-free first tier resolves the most frequent stall, and any failure returns control to the reactive controller. Our system reaches 68.75% SR on HM3D v1 \texttt{val} and 47.29% on MP3D \texttt{val}, the highest success rate among the zero-shot methods compared here. Choosing among far frontiers by argmax instead of deliberating costs 3.40 SR points in a paired comparison over all 2000 HM3D episodes (95\% CI [1.70, 5.05]); deliberating over every frontier does not recover them.