Humanoid teleoperation system improves long-term spatial awareness and control
SPOT: Spatial Perception-Oriented Long-Horizon Humanoid Teleoperation
Robotics
Summary
High-quality data is important to teach robots how to move and work like humans, but collecting this data can be hard. The authors created SPOT, a system that helps human operators see more around a robot using special cameras and displays, so they don’t miss important things while controlling the robot. Unlike usual systems, SPOT lets people look around freely without moving the robot’s head, making it easier to focus and control. Tests showed that SPOT helps people do complex tasks faster and more accurately with humanoid robots. This system could make it easier and better to train robots for complicated jobs over long periods.
humanoid robotsteleoperationspatial awarenessrobot perceptionfisheye camerastereoscopic displayegocentric viewvirtual realityrobot controldata collection
Authors
Lixing Fang, Ziyan Xiong, Sunli Chen, Zhiyang Dou, Chuang Gan
Abstract
High-quality demonstration data is becoming a central bottleneck for training general-purpose humanoid robots. While recent humanoid teleoperation systems have made substantial progress in retargeting human motion to robot motion, long-horizon loco-manipulation requires another capability: operators must maintain task-relevant spatial awareness over time, e.g., object locations, surrounding environments, the robot's pose. We call the extent of this awareness the operator's perceptual horizon. However, existing methods often shorten this: narrow views miss peripheral events, robot-mounted cameras become unstable during locomotion, and coupled head-view control makes looking around interfere with robot motion. We present SPOT, a Spatial Perception-Oriented VR Teleoperation system for collecting long-horizon humanoid demonstration data by providing extended perceptual horizon. SPOT combines a robot-mounted binocular fisheye camera, a wide-field stereoscopic display, viewpoint-decoupled free-looking, and visual stabilization to provide a robot-centric view that is wide, stable, and actively inspectable. Unlike conventional egocentric interfaces, SPOT decouples visual exploration from robot actuation: the egocentric stereo observation is rendered on a virtual hemisphere around the operator, so natural head rotations change where the operator looks within the wide-field view rather than commanding the robot head, camera, or torso. We evaluate SPOT on perception-critical humanoid data-collection tasks spanning drop recovery, peripheral retrieval, large-workspace bimanual manipulation, fine alignment, and dynamic interaction. SPOT improves efficiency, accuracy, and recovery speed, demonstrating its effectiveness for user-friendly and scalable long-horizon humanoid data collection.