Tactile sensing improves robot grasping after object moves
DA-GRD: Decision-Aware Grasp-Relevant Disambiguation for tactile recovery under perception-to-execution mismatches
Robotics
Summary
Robots sometimes see an object and plan how to grab it, but if the object moves before the robot acts, the planned grasp might not work. The authors developed a method called DA-GRD that uses touch to figure out which grasp will still work without needing to see the object again. DA-GRD picks where to touch next to narrow down possible object positions and stops once it finds a safe grasp to try. Experiments show this method helps robots successfully pick up objects even when visual information is outdated.
What this means in practice
- •For industrial robotics teams: Enable robots to recover effective grasps when objects shift without relying on additional vision, reducing task failures during automated handling.
- •For warehouse automation engineers: Improve robotic pick-and-place reliability by using minimal touch probes to confirm grasp strategies after items move during sorting or packing.
- •For medical device developers: Design tactile-aware robotic assistants that maintain reliable grasping of surgical tools despite unexpected object shifts without extra imaging.$Commercial implications: This paper enables tactile-guided grasp recovery in medical robots crucial for safe and precise tool handling.
Authors
Haoran Wang, Yuteng Sun, Yuanjie Li, Ruofei Bai, Meng Yee, Chuah, Wenyu Liang, Jun Li, Wei-Yun Yau
Abstract
Grasping is a fundamental robotic capability that bridges perception and physical task execution. This paper studies grasp pose recovery under a perception-to-execution mismatch, where a grasp generated from visual perception may become spatially stale if the object moves before execution, using only sparse tactile interactions and no further visual observations. We propose DA-GRD, Decision-Aware Grasp-Relevant Disambiguation, which maintains a weighted planar belief over possible object configurations and selects tactile probes according to their ability to eliminate hypotheses and improve agreement among candidate task grasps. Rather than fully relocalizing the object, DA-GRD stops when the remaining hypotheses support a common executable grasp. In MuJoCo experiments on ten rigid objects with translations up to 5~cm and yaw perturbations up to $\pm45^\circ$, DA-GRD achieves an 84.7% physical lift success rate, compared with 9.1% for stale AnyGrasp, 21.2% for the original fix-scan baseline, and 63.7% for fix-scan method adapted with an SE(2) belief. DA-GRD also achieves a 57.3% Task conditioned Success rate. Across objects, it uses a success-average of 4.13 tactile probes over the ten per-object means, corresponding to a 72.5% reduction relative to the fixed 15-probe baselines. Real-world experiments on six objects achieve 71.7% physical lift success and 38.3% task-conditioned success with 4.20 probes on average. These results show that tactile sensing can recover task-relevant grasps under vision-off conditions with limited physical interaction, without requiring complete object localization.