Robots learn to flatten and fold diverse clothes from simulation data
ClothDojo: Learned Data Generation and Benchmarking for Bimanual Garment Flattening and Folding
Robotics
Summary
Robots find it hard to flatten and fold clothes because cloth changes shape when touched. The authors created a simulation platform called ClothDojo that teaches robots how to move and handle many different clothes by watching examples and practicing in the computer. This system creates thousands of good examples of folding and flattening clothes, which helps train robots to see and manipulate clothes better. The trained robots do better than ones just trained on human examples, but still struggle with new clothes they haven't seen before.
What this means in practice
- •For roboticists: Train robots to flatten and fold a variety of garments more effectively using scalable simulated examples.
- •For industrial automation teams: Develop automated laundry or textile handling systems that adapt to different cloth types and appearances.
Authors
Zilong Huang, Zipeng Ye, Caicheng Wang, Yuchen Xie, Jiaxiang Luo
Abstract
Garment flattening and folding require robots to identify garment structure during manipulation and adapt their actions as the cloth deforms. We present ClothDojo, a simulation platform that generates adaptive manipulation trajectories at scale across diverse garments and initial states and evaluates policies in closed loop. ClothDojo encodes each garment's canonical mesh coordinates as an NOCS texture, providing canonical surface-position cues through the visual interface of a pretrained vision-language-action model. We fine-tune the model on human demonstrations and corrections to obtain a data-generation policy. The policy autonomously generates simulated manipulation trajectories, and we retain rollouts that satisfy task-specific success criteria. Each retained trajectory is re-rendered under a randomly sampled appearance to provide RGB observations for downstream policies that do not receive NOCS. Across both tasks, the policy produces nearly 8,000 successful trajectories over hundreds of garments. RGB policies trained on these generated trajectories outperform those trained directly on the original human demonstrations and corrections across all evaluated garment splits and appearance settings. Evaluations across multiple policy models show that overall performance and appearance robustness vary across models and tasks, while generalization to unseen garments remains a common challenge.