Bimanual robot learning improves by separating task from timing
Learn the Task, Not the Schedule: Reducing Spurious Cross-Arm Coupling in Bimanual Imitation Learning
RoboticsArtificial IntelligenceMachine Learning
Summary
Robots with two arms can learn tasks from demonstrations, but often pick up timing details that don’t actually matter, causing unnecessary coordination problems. The authors created a method called Constraint-preserving Temporal Recombination (CTR) that keeps the important task constraints but changes timing schedules during training. This helps robots focus on what each arm should do without being fooled by when the actions happen. Their tests show CTR improves success rates and reduces mistakes in both simulated and real-robot tasks.
What this means in practice
- •For robotics engineers: Improve training of dual-arm robots to perform tasks accurately regardless of action timing variations.
- •For industrial automation teams: Deploy more adaptable bimanual robots that handle flexible coordination without retraining for specific timing sequences.
Authors
Shuyuan Hu, Deng-Ke Wu, Yifan Jin, Qianhang Luo, Mingchen Ju, Kai Ye, Bo Zhao
Abstract
A bimanual demonstration specifies both what the robot should do and one particular schedule for when each arm should act. Standard imitation learning does not distinguish task-required cross-arm constraints from incidental scheduling choices, and may therefore exploit opposite-arm progress as a shortcut cue, creating spurious cross-arm coupling even where relative timing is irrelevant for the current task phase. We introduce Constraint-preserving Temporal Recombination (CTR), a training-time data intervention that preserves recorded arm-local sensorimotor content while retiming the corresponding segments to realize admissible schedules that preserve task-required cross-arm constraints. We use CTR to train Diffusion Policy, Action Chunking with Transformers, and $π_{0.5}$, and evaluate them on four simulated and four real-robot tasks. Across six real-robot bimanual progress-state initializations, CTR raises worst-state success from 56% for the strongest non-CTR schedule mixture to 76%. With $π_{0.5}$ on an Independent task, a 4 s pause transfers 2.56 s of delay to the opposite arm under the Concurrent schedule distribution, but only 0.25 s with CTR. On real robots, CTR violates task constraints in only 0--4% of trials, versus 36--98% for narrow-schedule baselines. Code will be released at https://github.com/MINT-SJTU/CTR.