Pace module improves cross-floor robot navigation success rates

Affordance-Conditioned Decision Making: Bridging the Semantic-Spatial Gap in Zero-Shot Cross-Floor Vision-and-Language Navigation

RoboticsArtificial Intelligence

Summary

Robots that navigate by following language instructions often struggle when moving through tricky places like staircases or narrow doorways because they don’t always know exactly how to physically move through them. This paper presents PACE, a technique that helps robots find exact spots they can safely pass through, linking language commands to real-world movement. The authors tested PACE with several navigation systems and showed it reliably improves success rates when robots move between floors or complex spaces. They also demonstrated its use in real-world settings, showing its potential to help robots handle unfamiliar environments better.

What this means in practice

  • For robotics engineers: Enhance indoor robots’ ability to navigate staircases and narrow passages based on language instructions by integrating PACE with existing navigation planners.
  • For autonomous delivery teams: Improve success rates of robots delivering packages across multiple floors in complex building environments using PACE’s affordance-based execution module.
  • For assistive robot designers: Create assistive home robots that better understand and physically execute language commands in cluttered or constrained spaces by using PACE’s corrective affordance grounding.$Commercial implications: Enables reliable navigation features in assistive robots for home use, improving physical interaction in complex spaces, making sale and deployment feasible.

Authors

Xuekang Yang, Lu Chen, Shuang Luo, Jialing Zhu, Qi Zhang, Yue Gao, Xiang Zhang

Abstract

Vision-and-language navigation increasingly relies on general-purpose semantic planners, yet translating correct high-level intent into reliable physical execution remains difficult in spatially constrained transitions. Reaching a staircase, doorway, or narrow passage does not ensure traversal; the agent must identify an executable affordance pose and recover from accumulated action errors. We propose PACE (Preference-refined Affordance-Conditioned Execution), a supervised local execution module that augments frozen zero-shot semantic planners for reliable cross-floor navigation. PACE grounds transition-related semantics into a long-horizon, agent-centric traversable affordance pose and conditions short-horizon action generation on this spatial target, thereby aligning semantic goals with physical execution. We further post-train PACE through failure-aware preference refinement using rollout-derived pairs that contrast normal or recovery behaviors with deviation-amplifying behaviors, thereby improving closed-loop correction. We integrate PACE into six open-source zero-shot VLN navigators and demonstrate consistent improvements on the cross-floor subsets of R2R-CE and RxR-CE, increasing the average success rate from 16.35% to 27.65% and from 4.76% to 12.06%, respectively. Real-world experiments further demonstrate PACE's applicability in unseen environments, highlighting the potential of traversable affordances to bridge semantic intent and reliable embodied behavior.
View PDFOpen arXiv