Humanoid robots learn tasks faster using feature-based perception
Data-Efficient Learning from Demonstration for Humanoid Robots via Feature-Level Perception and Prompt-Based Object Detection
Robotics
Summary
Teaching robots new skills often needs lots of video examples and powerful training methods, which can be slow and expensive. The authors created a way for humanoid robots to learn from fewer videos by focusing on important features like human movements and 3D object locations instead of full images. They combined special vision techniques and deep learning to predict robot movements, allowing robots to pick up tasks like picking and pushing with fewer examples. Their method works well both in computer simulations and on real robots.
What this means in practice
- •For robotics engineers: Design humanoid robots that learn manipulation tasks using fewer demonstration videos by applying feature-based perception and prompt-guided object detection.
- •For industrial automation teams: Integrate data-efficient learning methods into humanoid robot workflows to reduce training time and resource demands in pick-and-place and pushing operations.
Authors
Keyhan Rayati, AliReza Beigy, Anoosheh Saadati, Pourya Shahverdi, Mehdi Tale Masouleh, Ahmad Kalhor, Wing-Yue Geoffrey Louie
Abstract
Learning from Demonstration (LfD) allows robots to learn new skills from human examples; however, many vision-based methods require large datasets and resource-intensive end-to-end training. This paper introduces a data-efficient LfD framework for humanoid robots that learns manipulation skills from compact, task-specific features instead of raw images. The proposed approach integrates layered visual preprocessing, Kinect-based human joint estimation, prompt-based object detection, and depth information to extract human motion features and three-dimensional object locations from demonstration videos. These features are used to train a temporal deep learning model that predicts robot joint-angle trajectories for execution. The framework is implemented on a NAO humanoid robot, evaluated quantitatively in simulation, and demonstrated on physical hardware. In simulation, the robot achieves a 78% success rate on a pick-and-place task using only 16 demonstrations, which increases to 86% when trained with 100 demonstrations. The framework also achieves a 71% success rate on a pushing task using eight demonstrations, highlighting its effectiveness for data-efficient humanoid skill acquisition.