Benchmarking unified controls for seven dexterous robot hands
DexJoCo-X: Benchmarking Action Representations for Multi-Hand Dexterous Manipulation
Robotics
Summary
Robotic hands come in many shapes and types, making it hard to train a single control method that works well for all of them. The authors created a new testing setup called DexJoCo-X that lets them compare different ways to teach robots how to move multiple hands on different tasks. They found that using a combination of shared knowledge, a common action format, and specialized experts for each hand outperformed other methods. This means future robot hands could be controlled by one smart system instead of separate programs for each hand type.
What this means in practice
- •For robotics engineers: Develop a single control system that efficiently manages multiple robotic hand designs across diverse tasks to improve adaptability and reduce retraining time.
- •For automation system developers: Integrate unified action representations to enable multi-hand robotic systems in complex automated assembly or handling lines with varied grippers and manipulators.
Authors
Xiangwei Jiang, Yao Mu, Lixin Duan, Wen Li
Abstract
As dexterous hands proliferate, collecting data and training policies separately for every morphology becomes increasingly impractical. Scalable cross-embodiment learning therefore requires a unified representation that captures shared manipulation structure while preserving morphology-specific control. Differences in hands, tasks, datasets, and control interfaces prevent existing studies from isolating the effects of representation, pretraining, and architecture. We introduce DexJoCo-X, a benchmark and toolkit for controlled comparison across seven representative dexterous hands, six single-arm and bimanual tasks, and 2,100 balanced demonstrations. DexJoCo-X provides a matched multi-hand, multi-task protocol with common scenes, success criteria, and execution interfaces, redesigned glove-to-hand mappings, and an automated pipeline that expands reviewed demonstrations across randomized scenes. Using $π_{0.5}$, Ego-Pi, and Being-H0.5, we examine whether a shared action interface is sufficient for multi-hand learning. Expanding $π_{0.5}$ to an 80-dimensional bimanual output yields near-zero success. Ego-Pi preserves the pretrained action head through interleaved prediction and supports per-hand multi-task learning, but remains ineffective for seven-hand joint training. By contrast, Being-H0.5 combines cross-embodiment pretraining, a unified action space, and embodiment-aware experts, enabling one policy to control all seven hands. Within this architecture, function-aligned action slots achieve 47.7% mean success, compared with 47.0% for native coordinates and 33.1% for DexLatent. These results show that cross-embodiment representation depends on the entire learning system: action coordinates, pretraining, and architecture must jointly separate shared manipulation structure from embodiment-specific control.