Tactile encoders vary by task for robot touch sensing
TACTIC: Understanding Tactile Encoders and Conditioning for Contact-rich Robot Manipulation Policies
Robotics
Summary
Robots need touch sensing to handle objects carefully. The authors studied different ways to let robots feel through vision-based tactile sensors and compared many methods in real-world tests. They found that no single approach works best for all tasks. Instead, which method works best depends a lot on the exact job the robot is doing. This helps people build better robot touch systems by choosing the right method for their task.
What this means in practice
- •For industrial automation teams: Design robot touch sensing based on task to improve grip and manipulation accuracy in manufacturing.
- •For robotics software developers: Choose specific tactile encoder architectures to build or improve robot control policies for diverse contact tasks.
Authors
Seongjin Bien, Débora Oliveira Makowski, Carlo Kneissl, Reihaneh Mirjalili, Pankhuri Vanjani, Rudolf Lioutikov, Gitta Kutyniok, Florian Walter, Wolfram Burgard
Abstract
Tactile information is essential for contact-rich manipulation tasks in robotics. Vision-based tactile sensors make it particularly easy to design end-to-end manipulation policies with tactile sensing, as they enable the use of existing encoders from computer vision. However, this has led to a huge variety of architectures, training datasets, and evaluation protocols, making it difficult to determine which design choices best encode touch. In this work, we address this gap and present a comprehensive study of tactile encoders and fusion strategies across various contact-rich manipulation tasks in real-world experiments. To enable a controlled comparison, we train and evaluate all models under the same pipeline and experimental setup, comprising more than 2000 real-world rollouts. Our results go beyond other studies that only compare simulation performance, which does not necessarily translate to real-world settings, where large-scale evaluations are needed to obtain reliable statistics. Our key finding is that there is no universally optimal representation or fusion strategy for encoding visual-tactile. Instead, the best encoder backbone and fusion scheme depend strongly on the task.