Large synthetic dataset enables robots to fold and unfold t-shirts
FoldNet++: a Large-Scale Synthetic Dataset for Robotic T-Shirt Folding and Unfolding
Robotics
Summary
Folding and unfolding T-shirts is hard for robots because clothes are soft and change shape easily. The authors created a huge fake dataset with many different T-shirts, environments, and robot types to teach robots how to handle this task. They used special points marked on the T-shirts to guide the robot actions and trained computer programs to do the folding. Their programs, trained only on this fake data, worked well when tested on real T-shirts the robots had never seen before.
What this means in practice
- •For robotics engineers: Develop robotic systems that can fold and unfold various T-shirts using policies trained with a large-scale synthetic dataset.
- •For automation integrators: Create automated laundry folding solutions that adapt to unseen T-shirt designs and initial configurations by utilizing the dataset-trained policies.$Commercial implications: Enables sale of adaptable laundry-folding robots for commercial laundries and apparel businesses, improving efficiency and reducing manual labor.
Authors
Yuxing Chen, Zhiyuan Wei, Bowen Xiao, Zhizheng Zhang, He Wang
Abstract
Due to the highly deformable nature of garments, training a generalizable policy for robotic T-shirt folding and unfolding remains a significant challenge. In this work, we present a large-scale synthetic dataset for robotic T-shirt folding and unfolding, covering 6 robotic embodiments, 1K T-shirts, 1K environmental assets, and 120K episodes with rich annotations, which can be used to train a wide range of manipulation policies. We first follow the FoldNet pipeline to generate a large-scale dataset of physically simulatable T-shirts with diverse appearances and annotated semantic keypoints. Based on these semantic keypoints, we then generate manipulation demonstrations for different robotic embodiments through a unified rule-based framework. We use these demonstrations to train visuomotor policies, and experimental results demonstrate that models trained solely on our synthetic data can achieve over 90\% end-to-end task success rates when directly deployed to unseen real-world environments and previously unseen T-shirts from arbitrary initial configurations. Project URL: https://pku-epic.github.io/FoldNetXX/.