Bimanual tactile learning improves wrist motion and force recognition
BiView-Touch: Learning Bimanual Tactile Representations by Cross-Hand Completion
Computer Vision and Pattern RecognitionRobotics
Summary
Touch sensors on both hands can feel the same physical interaction from different angles, but previous methods mostly studied each hand alone. The authors created BiView-Touch, which learns to predict the full feeling on one hand using partial information from that hand plus the other hand's sensing data. This approach captures how the two hands' sensations relate over time and body layout, leading to better understanding of hand motions and forces with very few labeled examples. They also introduced a new dataset with many tactile tasks and showed their method works across different tasks and recording sessions.
What this means in practice
- •For robotics engineers: Improve robot hand control by learning tactile signals from both hands together for better motion and force recognition with minimal labeled data.
- •For prosthetics designers: Design prosthetic hands that interpret coordinated touch signals from both hands to enhance interaction phase recognition.
Authors
Chenxin Liang, Youchen Lai, Chuqiao Lyu, Tianxing Chen, Shoujie Li, Wenbo Ding
Abstract
Bimanual interaction produces complementary tactile views of the same physical process, yet existing tactile representation learning largely models the two hands independently or combines them only for downstream prediction, leaving their cross-hand relationship unexplored. To exploit this overlooked structure, we introduce BiView-Touch, a tactile-only framework that completes masked target-hand latents from the remaining visible target-hand regions and the synchronized full contralateral hand. A student encoder with a geometry-conditioned directional decoder predicts full-view EMA latent targets, while temporal and layout counterfactuals encourage sensitivity to synchronized and anatomically organized source information. Controlled ablations and source-context interventions show that BiView-Touch learns structured cross-hand dependence on temporally aligned and anatomically organized contralateral tactile context, rather than benefiting from bilateral input alone. On the public HumanTouch dataset, its frozen representations consistently outperform representative self-supervised baselines across low-label settings. With only 5\% downstream labels, BiView-Touch achieves relative balanced-accuracy gains of 7.1\% on bilateral wrist-motion recognition and 14.1\% on force-derived interaction-phase recognition. We further introduce BVT-20, a 20-task bilateral tactile dataset, and demonstrate transfer across recording sessions and pretraining corpora, including transfer to a held-out bimanual task. Our code and dataset details are available on the anonymous project page: https://anonymous.4open.science/w/biview-touch-review-site-050C/.