Video2DoorTraversal: Push Door Traversal via Simulated Door Twins
2026-08-20 • Robotics
Robotics
AI summaryⓘ
The authors created a system called Video2DoorTraversal that helps robots open and go through doors by learning from just one video of a real door. Their method makes a virtual copy of the door, uses simulation to practice door-opening motions, and then trains a robot to perform these actions. The robot can then open and pass through doors with high success, even for doors it has never seen before. All the processing happens on the robot itself, enabling quick and effective door navigation.
loco-manipulationarticulated objectsrobot simulationpolicy learningrobot perceptionmobile manipulatorsrobot controlzero-shot learningreal-to-simsim-to-real
Authors
Xincheng Tang, Yiji Chen, Youhan Xie, Wanyu Li, Zhengjie Shu, Lai Jiang, Wenkang Hu, Yitong Li, Jinchuang Zhang, Xibin Song, Ruigang Yang
Abstract
Door opening and traversal is a long-horizon loco-manipulation task that requires precise handle interaction and coordinated base-arm control. We present Video2DoorTraversal, a single-video real-to-sim-to-real framework for wheel-legged mobile manipulators. Given one RGB video of a real door, DoorTwin reconstructs an instance-aligned, articulated, and simulation-ready door twin with realistic geometry and appearance. A simulation-in-the-loop agent converts the recovered articulation into a parameterized skill program and iteratively refines failed rollouts to generate physically executable demonstrations. These demonstrations are used to train ArticuACT, a dual-depth policy that predicts coordinated base, arm, and gripper commands using robot-centric camera conditioning and interaction-aware supervision. With all perception and policy inference running onboard, the system achieves a 96.57% average success rate across five real doors and an 80.95% zero-shot success rate on structurally similar unseen doors, while completing the full approach, opening, and traversal sequence in approximately 13s on average. Project Page: https://video2doortraversal.github.io/.