Robotic hand rotates balls blind using touch only
No Eyes, No Problem: What Does Touch Contribute to Blind Baoding Ball Manipulation?
Robotics
Summary
Humans can easily manipulate small objects like Baoding balls using only their sense of touch, but robots find this very difficult without vision. The authors trained a robot hand in simulation to rotate Baoding balls without seeing them, relying only on touch sensors and joint positions. They made key improvements to better match the real robot's sensing and motion, allowing the robot to do many rotations continuously on a physical hand. They also studied which touch sensors helped the most, showing touch alone can guide complex object manipulation.
What this means in practice
- •For robotics engineers: Develop dexterous robot hands capable of sustained object manipulation without any visual input using touch and proprioceptive sensing.
- •For robot manufacturers: Improve reliability of robots performing blind tasks like assembly or handling fragile objects by incorporating tactile-driven control policies validated on real hardware.$Commercial implications: Enables production robots to operate safely and reliably without cameras, benefiting industrial automation companies producing tactile-enabled robotic systems.
Authors
Ayush Deshmukh, Nalin Agarwal, Elle Miller, Sethu Vijayakumar
Abstract
Humans manipulate objects within their hands through touch alone with remarkable ease, but for robots, achieving sustained real-world manipulation without vision remains an open challenge. Although reinforcement learning (RL) can generate sophisticated blind manipulation policies in simulation, transferring them to a real dexterous hand remains hindered by continuously changing contacts, motion-induced tactile information drift, and the gap between simulated and physical dynamics. We study this problem through Baoding ball rotation on the Shadow Dexterous Hand Lite, using only proprioception and binary tactile feedback as observations. We achieve successful zero-shot transfer through a combination of a sensor-aligned, coupling-aware embodiment in simulation, tactile calibration, and two novel episodic domain randomisations: slew-rate action limits and STuck-AT Taxels. On the physical hand, the policy achieves sustained blind Baoding rotation, completing 225 consecutive $\mathbf{180^\circ}$ rotations (112.5 full rotations) at $\sim0.36 \;\text{rot/s}$ in a single continuous trial without any vision or object-state information. Furthermore, we evaluate the contribution of tactile sensing relative to proprioception alone and identify the hand regions where tactile information is most important. This is the strongest reported hardware result for Baoding ball manipulation in both rotation count and speed, irrespective of the sensing modality used. By effectively bridging the sim-to-real gap, our framework sets a new benchmark for sustained tactile-driven dexterous manipulation on physical hardware.