Humanoid robot learns to navigate cluttered spaces using vision and language
TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model
RoboticsArtificial Intelligence
Summary
Navigating tight spaces with lots of obstacles is hard for robots that look like humans because they have many body parts that must move carefully together. The authors created TANGO, a system that helps a humanoid robot understand language instructions and see its surroundings to decide how to move its arms, body, and legs all at once. They trained TANGO in computer simulations where it learned to avoid obstacles and keep moving smoothly. When tested, the robot followed spoken directions and moved safely through messy rooms even without practicing in real life first.
humanoid robotnavigationvision-language modeljoint-space actionstrajectory planningrobot kinematicsreinforcement learningcollision avoidancesimulation trainingegocentric RGB
Authors
Anqi Li, Yuxin Chen, Zhaobo Li, Zhuo Cao, Junli Ren, Masayoshi Tomizuka, Dhruv Shah
Abstract
We study the problem of navigating cluttered indoor environments with a humanoid robot. Unlike conventional methods that model navigation as a 2D path planning problem, humanoid traversal in cluttered environments requires continuous geometry-aware whole-body adaptation, including coordinated arm placement, torso adjustment, and gait modulation for collision-free movement through complex 3D spaces. We introduce TANGO, the first whole-body vision-language navigation framework for language-conditioned humanoid traversal in cluttered environments. Given a natural-language instruction and egocentric RGB observations, TANGO directly predicts 29-DoF joint-space actions for downstream whole-body control. We train TANGO entirely in simulation by synthesizing diverse collision-free traversal behaviors via global path planning, kinematic whole-body motion generation, obstacle-aware motion editing, and RL-based tracking. This pipeline provides dynamically feasible action supervision for learning language-conditioned whole-body policies. In extensive simulation experiments, TANGO demonstrates state-of-the-art performance in vision-language navigation, while outperforming strong modular baselines in navigating challenging scenes requiring obstacle negotiation. Lastly, we deploy TANGO zero-shot on a Unitree G1 humanoid robot, and observe robust language-guided traversal in cluttered real-world scenes without training on any real-world navigation data.