Visible touch improves robot grasping using magnet sensors
Visible Touch: Rendering Contact for Visuomotor Policies
Robotics
Summary
Robots struggle to manipulate objects reliably because they often rely only on vision, missing the sense of touch that humans use naturally. The authors show that by adding touch information in a way that overlays the contact signals directly onto the robot’s visual input, existing AI models can understand and use touch without changing their design. They built a low-cost magnetic touch sensor that attaches to robots and demonstrated that it significantly improves the success of robot learning in manipulation tasks. This approach works well with current vision-based AI models and is openly available for others to build.
What this means in practice
- •For robot engineers: Equip robot arms with magnetic touch sensors to enhance manipulation success using existing vision-based AI models without redesigning their software.
- •For automation integrators: Deploy low-cost tactile sensors alongside cameras to improve reliability in automated pick-and-place and assembly tasks involving physical contact.
Authors
Metin Alp Dogan, Edward Sun, Feng Xu, Daniel Wu, Allen Peng, Dennis Hong, Yuchen Cui
Abstract
Integrating contact information into visuomotor policies remains an open problem. Touch is essential to robust manipulation, yet most modern policies, including pretrained vision-language-action (VLA) models, operate from vision and proprioception alone. Existing approaches to closing this gap require specialized tactile hardware, add separate tactile encoders, or commit to non-image policy backbones, all incompatible with the modern paradigm of image-conditioned policies built on pretrained 2D visual representations. Our key insight is that the bottleneck is not the contact information itself, but how it is delivered: when contact signals are exposed in the same spatial frame as the scene the policy already attends to, they become directly usable by any image-conditioned policy without architectural changes. We operationalize this insight in Visible Touch, paired with a custom low-cost magnetic contact sensor that is open-sourced and fabricated from off-the-shelf parts via a parametric CAD-to-mold pipeline. Across the LIBERO benchmark, Visible Touch improves BC-Transformer success by 15.7 percentage points on average in the 2-view setting, with similar gains in the 1-view setting; controlled comparisons show that the contact-integration strategy strongly affects how effectively tactile information is used. The pattern holds when fine-tuning pretrained VLAs: miniVLA on LIBERO gains 25 percentage points on average, and $π_{0.5}$ on four real-world contact-rich tasks gains 30 percentage points with our custom sensor.