Pixel system integrates robot hardware and learning for versatile mobile tasks
Pixel: A Co-Designed Full-Stack Robot System for Learning General-Purpose Mobile Manipulation
RoboticsArtificial IntelligenceComputer Vision and Pattern RecognitionMachine Learning
Summary
Getting robots to move around and handle many kinds of objects is hard because the robot’s body, how people teach it, and how it learns often don't fit well together. The authors created Pixel, a system that carefully designs robot parts, data collection from people, control methods, and learning all as one unit. This lets the robot do tricky tasks that involve moving, seeing, picking up things, and working with its whole body, all guided by language commands. Their work shows how building these elements together can help robots learn to do many mobile tasks better.
What this means in practice
- •For robotics engineers: Build and test mobile robots that can learn complex manipulation skills from human demonstrations and language commands.
- •For automation integrators: Deploy low-cost mobile robots capable of adaptive manipulation for versatile tasks in environments like warehouses or homes.$Commercial implications: Enables commercial mobile manipulation robots with improved dexterity and adaptability using integrated hardware and learning.
Authors
Tony Tao, Jason Jingzhou Liu, Ruslan Salakhutdinov, Deepak Pathak
Abstract
General-purpose mobile manipulation requires a combination of capable robot hardware, scalable demonstration collection, expressive whole-body control, and effective policy learning. Yet these components are often developed independently, creating mismatches that limit the capabilities of the complete system. We present Pixel, a co-designed full-stack system for learning general-purpose mobile manipulation. Pixel combines a low-cost wheeled humanoid with human-aligned kinematics and compliant grippers, multiple teleoperation and robot-free human data-collection interfaces, a unified whole-body controller, and a common policy learning pipeline across human and robot demonstrations. A hierarchical learning stack further enables language-conditioned visuomotor control and long-horizon task execution. We evaluate the system across hardware, data collection, and policy learning, and demonstrate mobile-manipulation behaviors requiring dexterity, active perception, navigation, and whole-body coordination. We hope that Pixel provides an integrated and extensible foundation for studying how robot hardware, data, control, and learning can be designed together for robot learning. More results at https://pixel-robot.com/pixel