Vision sensor separates texture and force signals for better robot touch
DeTac: Dual-branch Decoupling of Texture and Force Signals in Vision-Based Tactile Sensors
Robotics
Summary
Robots use special cameras to 'feel' objects, but the signals showing texture and force often get mixed up. The authors created DeTac, a system that splits these signals into two clear images, making it easier for robots to understand what they are touching. They built new hardware and collected real data to train and test their method. This improvement helps robots better judge surfaces and forces when grabbing things.
What this means in practice
- •For robotic manipulation teams: Improve dexterous robotic grasping by separately identifying surface texture and contact force from tactile sensor images.
- •For visual sensor developers: Create vision-based tactile sensors with better signal clarity to aid object recognition and force measurement.
Authors
Zhirui Zhang, Yunkai Tang, Jiahang Li, Fan Fei, Jinnuo Zhang, Kairan Ding, Jieji Ren, Yandong Guo, Boxin Shi
Abstract
Vision-based tactile sensors enable fine-grained perception for dexterous robotic manipulation, but current marker-based designs tightly couple surface texture and contact force signals within a single observation. Existing data-driven decoupling methods rely on serial mask-based inpainting, which suffers from mask-sizing dilemmas caused by physical indentation effects, error retention, and lack of real paired data for supervision. To address these challenges, we propose DeTac, a novel dual-branch generative framework that concurrently extracts clean shading and marker images from a single mixed tactile image. DeTac leverages full-image synthesis to naturally bypass mask dependency and introduces a cross-branch synergistic optimization that enables mutual supervision, effectively preventing error retention. Furthermore, we construct a strictly aligned, real-world paired tactile dataset using custom hardware to provide high-quality supervision. Extensive experiments demonstrate that DeTac achieves state-of-the-art decoupling accuracy, significantly facilitating downstream surface reconstruction and force estimation tasks while exhibiting cross-sensor generalization.