TACROSS enables efficient human-to-robot touch learning with low-cost gloves
TACROSS: An Efficient and Low-Cost Scalable Human Touch System Across Heterogeneous Tactile Sensors for Dexterous Robot Learning
RoboticsMachine Learning
Summary
Teaching robots to understand and use touch is usually expensive and slow because robotic sensors are costly and hard to handle. The authors created TACROSS, a system that uses cheap human-wearable gloves with many touch sensors to record how people feel things. TACROSS matches these human touch signals with robot sensors by focusing on when and where contact happens, rather than the exact sensor readings. This approach helps robots learn complicated hand movements more quickly and cheaply by sharing data between humans and robots.
What this means in practice
- •For robotics engineers: Use scalable tactile data from low-cost human gloves to train robots on delicate hand manipulation faster and cheaper than direct robot demonstrations.
- •For manufacturing automation teams: Improve robots’ ability to handle complex touch-sensitive tasks by enabling efficient tactile data collection through human demonstrations with inexpensive gloves.
Authors
Bo Chen, Huanzhang Hu, Junyang Ma, Bo Yue, Fangdi Yu, Haijier Chen, Xianxin Lai, Shuyu Pan, Zhen Yang, Xiaoquan Sun, Wenze Cui, Zhongliang Jiang, Shaopeng Liu, Jiayu Chen
Abstract
Collecting tactile demonstrations on robots is costly and slow, motivating the use of lower-cost human tactile gloves for scalable data collection. However, human capacitive/piezoresistive gloves and robotic tactile sensors differ fundamentally in transduction principle, sensor layout, spatial resolution, and dynamic response, making alignment of raw sensor channels ill-posed. To address this problem, we present TACROSS, a scalable system for learning from human touch and transferring it to robots that bridges this heterogeneity by aligning tactile streams at the level of contact events rather than raw sensor values. The hardware component of TACROSS integrates a piezoresistive glove with five layers and a cost of USD 10.86 with 285 sensing points. To align contact semantics, we design canonicalizers and residual adapters that map heterogeneous signals into a shared tactile latent with 256 dimensions via a temporal Transformer with attention across fingers. We further introduce a robot-grounded policy learning scheme in which robot demonstrations provide the sole source of ground-truth action supervision, while human demonstrations support tactile representation learning and provide confidence-weighted auxiliary supervision through valid retargeted hand targets. We evaluate our system on four contact-rich manipulation tasks. Compared to conventional teleoperation, our proposed system achieves a 3.5-fold efficiency improvement while reducing demonstration acquisition equipment cost by 95.7%. We will open-source the TACROSS hardware and software system and publicly release a tactile dataset comprising over 150 hours of recordings. Project page: https://tacross-touch-project.github.io/.