Humanoid robot navigates cluttered spaces using vision and language
TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model
RoboticsArtificial Intelligence
Summary
Navigating complex indoor spaces is hard for humanoid robots because they need to move their whole body carefully to avoid obstacles. The authors created TANGO, a system that uses both what the robot sees and natural language instructions to plan its full-body movements. TANGO was trained in simulations to learn how to move without bumps and can directly control many robot joints to walk and avoid objects. The system worked well in tests and was even used on a real robot to navigate cluttered rooms without extra training on real-world data.
What this means in practice
- •For robotics engineers: Develop humanoid robots that follow spoken or written commands to safely move through cluttered indoor environments.
- •For warehouse automation teams: Use vision-language whole-body control models to enable robots to navigate complex storage areas with obstacles effectively.
Authors
Anqi Li, Yuxin Chen, Zhaobo Li, Zhuo Cao, Junli Ren, Masayoshi Tomizuka, Dhruv Shah
Abstract
We study the problem of navigating cluttered indoor environments with a humanoid robot. Unlike conventional methods that model navigation as a 2D path planning problem, humanoid traversal in cluttered environments requires continuous geometry-aware whole-body adaptation, including coordinated arm placement, torso adjustment, and gait modulation for collision-free movement through complex 3D spaces. We introduce TANGO, the first whole-body vision-language navigation framework for language-conditioned humanoid traversal in cluttered environments. Given a natural-language instruction and egocentric RGB observations, TANGO directly predicts 29-DoF joint-space actions for downstream whole-body control. We train TANGO entirely in simulation by synthesizing diverse collision-free traversal behaviors via global path planning, kinematic whole-body motion generation, obstacle-aware motion editing, and RL-based tracking. This pipeline provides dynamically feasible action supervision for learning language-conditioned whole-body policies. In extensive simulation experiments, TANGO demonstrates state-of-the-art performance in vision-language navigation, while outperforming strong modular baselines in navigating challenging scenes requiring obstacle negotiation. Lastly, we deploy TANGO zero-shot on a Unitree G1 humanoid robot, and observe robust language-guided traversal in cluttered real-world scenes without training on any real-world navigation data.