Manipulation skill dataset improves multi-room robot task success
ManiUnit: A Manipulation Skill Dataset and Benchmark for Long-Horizon Tasks
Robotics
Summary
Performing long tasks that involve moving around and manipulating objects is hard for robots, because small changes can confuse them or cause errors to build up. The authors created ManiUnit, a large dataset and testing system that breaks these tasks into smaller skills with clear instructions and checks each skill individually. This allows better training and evaluation by isolating problems and testing how well robots handle different starting positions. They show that skill-specific training improves performance on complex tasks compared to training on whole tasks directly, and that their method helps identify which skills need improvement.
What this means in practice
- •For robotics engineers: Train and evaluate robot manipulation skills individually to improve success on complex multi-room tasks using the ManiUnit dataset and benchmark.
- •For robotics research teams: Diagnose and improve specific manipulation skills in robots by isolating subtask performance and testing sensitivity to starting position variations with ManiUnit.
Authors
Guoting Wei, Dawei Yan, Xia Yuan, Gengming Zhang, Yelin He, Guodong Du, Jiaquan Ye, Heng Zhang, Xinming Wei, Xianbiao Qi, Chunxia Zhao, Haokui Zhang, Rong Xiao
Abstract
Long-horizon mobile manipulation requires a robot to navigate multi-room environments and execute a sequence of manipulation skills under a single natural language instruction. Learning and evaluating these skills present three challenges: similar observations under a fixed task instruction may make skill selection ambiguous; even when a preceding skill succeeds, the robot state inherited by the next skill may deviate from its demonstrated starting states and affect execution; and task-level metrics hinder skill-specific diagnosis, while early failures leave later skills untested. We therefore introduce ManiUnit, a manipulation skill dataset and benchmark built from 50 BEHAVIOR-1K activities. Its dataset contains 137,899 segments across 21 skill types and 417 subtasks, and its benchmark contains 1,260 test instances. Correspondingly, ManiUnit pairs each segment with an explicit subtask instruction; measures sensitivity to perturbations of the robot's starting base position or joint configuration; and restores intermediate simulator states and defines local success conditions so that each skill can be evaluated without executing preceding stages. Evaluations of representative vision-language-action (VLA) policies show that similar aggregate scores can hide substantial per-skill differences. The tested starting-state perturbations also degrade execution: on the full benchmark, joint perturbations reduce success rates by approximately 56% relative to those from demonstrated starting states. On two long-horizon activities, a skill policy trained on ManiUnit segments achieves 78.7% local manipulation success, compared with 49.3% for a task policy trained on complete demonstrations. The trained skills further support complete-task execution on these activities, as coordinating the task and skill policies through a planner raises full-task success from 4.0% to 18.0%.