DenseHap improves hand contact and pressure detection from single images
DenseHap: Dense Hand Haptic Estimation from Monocular Images
Computer Vision and Pattern Recognition
Summary
Understanding how a human hand touches and presses objects is important for robots that handle things skillfully. The authors present DenseHap, a new method that can look at a single photo of a hand and figure out how close the hand is to objects, where it is touching them, and how much pressure it exerts. This approach combines several tasks into one system and improves accuracy compared to earlier methods. It uses a special dataset and a 3D hand skeleton to learn better hand and contact details from images.
What this means in practice
- •For robotics engineers: Improve robotic grasping by estimating detailed hand contact and pressure from monocular images during human demonstrations.
- •For computer vision developers: Build better augmented reality systems that detect hand-object interactions precisely from a single camera feed.
Authors
Junhan Xue, Rolandos Alexandros Potamias
Abstract
As dexterous robotic manipulation advances, scalable annotations of human hand-object interactions are becoming increasingly important. Furthermore, previous hand contact estimation methods suffered from issues such as dorsal-side misclassifications and insufficient precision, while lacking a unified framework for integrating multiple hand to environment interaction modalities, including distance fields, contact thresholds, and pressure. To address that, we present DenseHap, a unified end-to-end approach to estimate hand distance field recognition, comprehensive hand contact detection, hand pressure analysis, and hand reconstruction from a single monocular image. By incorporating three distinctly defined heads, along with a pretrained 3D reconstructed skeleton and an enhanced dataset, DenseHap outperforms previous state of the art models on four evaluation datasets, improving the F1 score by 8.5 percentage points (a 16% relative improvement) on untrained datasets and by 3 percentage points (5% relative improvement) on in-distribution datasets. Furthermore, DenseHap has effectively improved short-range hand estimate and pressure evaluation. Code and models will be publicly available upon the paper's acceptance.