SlipSense uses touch and vibration for fast slip detection in robots
SlipSense: Multimodal Tactile Learning for Low-Latency and Generalized Slip Detection
RoboticsArtificial IntelligenceMachine Learning
Summary
Slip detection helps robots hold and move objects without dropping them. The authors built SlipSense, a system that uses both pressure sensors and tiny accelerometers to feel when an object starts to slip. By combining these two types of signals and analyzing them quickly, SlipSense detects slip events within about 23 milliseconds. It works well on many objects and even when used with a different robot without extra training.
What this means in practice
- •For robotics engineers: Equip robotic hands with SlipSense to detect and react to object slip quickly for safer and more reliable grasping.
- •For industrial automation teams: Integrate SlipSense-enabled sensors into automated pick-and-place machines to reduce object dropping and improve throughput.
Authors
Tong Jian, Aditya Thurvas Senthil Kumar, Xinyi Li, Ziling Chen, Tianyu Dai, Ali Sengul, Matteo Grimaldi, Wenjie Lu, Saleh Nabi, Tao Yu
Abstract
Slip detection is fundamental to dexterous manipulation, yet existing systems often lack precise characterization of detection latency and cross-platform generalization. We present SlipSense, a multimodal tactile slip-detection framework built on TacV5, a compact sensor integrating a $32 \times 32$ piezoresistive array operating at 240 Hz and a 3-axis MEMS accelerometer operating at 8 kHz. The piezoresistive array captures spatial pressure distributions, while the accelerometer captures friction-induced vibrations, providing complementary slip cues. The framework performs modality-specific encoding, intra-sensor fusion, and cross-modal attention with causal temporal prediction at 240 Hz. Experiments on a dataset of 1.4 million frames spanning 37 objects demonstrate the complementarity of the two modalities. SlipSense achieves 96.7% Macro F1 with a false-positive rate below 1.6%, detecting 76% of slip events within 23.1 ms. When trained solely on UMI data, SlipSense generalizes zero-shot to a Tesollo dexterous hand, transferring across unseen objects, distinct sensor units, and robotic platforms without retraining.