Robot learns to adjust grip on different flexible objects by guessing their material
Online Material Estimation for Conditioned Diffusion Policy in Shaping Deformable Linear Objects
Robotics
Summary
It is hard for robots to shape bendy objects like cables or ropes because they behave differently depending on what they are made of. The authors created a robot system that looks at the object and its own movement to figure out what material the object is. Then the robot uses this guess to decide how to move better to shape the object correctly. Their tests showed the robot can guess materials and succeed in shaping just as well as if it was told the exact material beforehand. The system sometimes mixes up very similar materials, which causes mistakes.
What this means in practice
- •For robotics engineers: Build robotic handlers that adjust their actions during manipulation by estimating material properties online for flexible objects.
- •For industrial automation teams: Improve automatic assembly or sorting systems by enabling robots to adapt motion for different flexible materials without prior material info.
Authors
Ryunosuke Yamada, Tomohiro Motoda, Yukiyasu Domae, Tokuo Tsuji
Abstract
Shape control of deformable linear objects (DLOs) is challenging for imitation learning because deformation behavior varies with material properties such as stiffness and elasticity, so a single policy must generate different action sequences for different objects even when the goal shape is identical. We propose a diffusion policy conditioned on material labels that are estimated online during manipulation. A recurrent estimation network predicts the material label of the grasped object from the time series of multi-view images and robot joint states, and the predicted label conditions the diffusion policy at every inference step. We collected 480 real-robot demonstrations covering four DLO materials and three groove-placement tasks, and compared per-material specialist policies, a task-conditioned policy without material labels, a policy conditioned on ground-truth material labels, and the proposed policy. Conditioning on ground-truth material labels improved the average success rate from 45.8% to 60.0% over the task-only policy, and the proposed policy reached 60.8% without any prior material information, matching the policy given ground-truth labels. A post-hoc analysis shows that the estimator extracts material-related information from the manipulation observations and that the diffusion policy responds to the resulting conditioning signal, while the one pronounced failure case is associated with persistent confusion between two similar materials.