Tactile contact improves robotic hand focus for precise object placement
TACIT: Tactile Contact Supervision for Spatial Attention in Dexterous Manipulation
Robotics
Summary
Robotic hands often struggle to adjust their movements when objects move around. The authors propose a new method called TACIT that uses touch sensors during training to teach the robot where to focus its attention visually. This method improves the robot’s ability to place objects correctly, even with very few demonstrations. TACIT shows much better success rates than previous methods in real-world robot tests.
What this means in practice
- •For robotics engineers: Train robotic manipulators to handle objects precisely with fewer demonstrations by using tactile feedback to guide visual attention during training.
- •For automated assembly teams: Improve robotic insertion and placement tasks in manufacturing by integrating tactile contact data to enhance spatial accuracy without additional manual labeling.
Authors
Yanhou Lai, Fucai Zhu, Ruiqiang Wang, Koichi Hashimoto
Abstract
Visuomotor policies trained from a few demonstrations may reproduce demonstrated trajectories without reliably following changes in object position. Existing approaches with explicit attention typically obtain spatial priors from human annotation or visual models. We introduce TACIT (tactile contact informs attention), which uses measured tactile contacts from teleoperated demonstrations to supervise spatial attention without additional point annotation. Gaussian targets over preceding camera point clouds supervise an attention head whose pooled output conditions a visuotactile diffusion policy. Targets are used only during training; tactile observations remain inputs at inference. In the primary real-robot benchmark, with ten demonstrations per task and five demonstrated placement regions, TACIT achieves 66.7% success on ball placement and 73.3% on peg insertion, compared with 10.0% and 20.0% for input-matched 3D visuotactile fusion and 20.0% and 43.3% for vision-only DP3. TACIT enters the 150 mm palm-to-object approach region within 12 seconds in all 30 trials per task; all remaining failures occur after arrival. Across three training seeds on real ball and simulated peg, TACIT outperforms input-matched fusion and an architecture-matched control without explicit attention supervision, supporting the contribution of supervision beyond branch capacity. Pre-contact and contact-time supervision show no consistent ordering. These results demonstrate that measured tactile contact provides effective spatial supervision for approach behavior from few demonstrations within the evaluated workspace.