Ego robot co training improves robot learning with human data
AtomEgo: Exploring Ego-Robot Integration for Embodied Foundation Model Pretraining
RoboticsArtificial Intelligence
Summary
Training robots to understand and act like humans faces challenges because robots move differently and have different abilities. The authors collected a large amount of human first-person video data and tried different ways to use it to teach robots better. They found that how well the human data helps depends on how well it is matched to the robot's abilities and how much data is used. This insight can help improve robot learning in the future.
What this means in practice
- •For robotics engineers: Improve robot learning by integrating large-scale egocentric human interaction data into pre-training pipelines.
- •For computer vision developers: Develop better models for vision-language-action tasks by combining human and robot data with alignment techniques.
Authors
Di Wu, Dongchen Zheng, Junhe Sheng, Zhongxing Wei, Songxin Zhang, Zejian Xie, Xiaoquan Sun, Junyang Zheng, Zhuoyang Song, Jiaxing Zhang, Jiayu Chen
Abstract
Embodied foundation models are constrained by the limited scale and diversity of robot demonstrations, motivating the use of large-scale egocentric human interaction data. However, how to effectively incorporate such data into embodied-model pre-training remains unclear because of substantial embodiment and action-space gaps between humans and robots. We present AtomEgo, a systematic study of ego--robot co-training supported by a curated corpus of approximately 2,659 hours and a scalable data processing pipeline. Across vision--language--action and world--action model architectures, we investigate three representative paradigms: joint co-training with domain-specific action heads, progressive ego-to-robot transfer through embodiment alignment, and joint video--action modeling. We evaluate these paradigms through multi-task real-robot experiments and language-conditioned cross-embodiment representation analysis. Our results reveal a simple principle: Data Scale * Alignment Quality --> Capability Gain; egocentric data can improve generalization, but their value depends on how effectively they are aligned and utilized. This principle can provide practical guidance for scalable ego--robot pre-training.