Physically stable human hand grasps reconstructed from single images
StableGrasp: Reconstructing Physically Stable Human Hand Grasps from Single Images
Computer Vision and Pattern RecognitionGraphicsRobotics
Summary
Figuring out how a hand is holding something in a photo is hard because not only the hand's shape but also the forces that keep the object stable must be guessed. The authors developed StableGrasp, a method that uses a computer simulation that can be fine-tuned to find hand poses and grip forces that not only look right but also physically support the object. This approach makes sure the reconstructed grip is much more stable when tested in simulations, while still matching the photo’s appearance and hand shape. Their method improves over others that either only look at the hand’s shape or use simpler physics that might not hold up.
What this means in practice
- •For robotic hand builders: Generate stable human-like grasps from single images to teach robots how to hold objects realistically.
- •For virtual reality developers: Create realistic and physically stable hand poses from images to improve immersion and interaction in VR applications.
Authors
Han Jiang, Etienne Vouga, Qixing Huang, Georgios Pavlakos
Abstract
Reconstructing a physically stable human grasp from a single RGB image is challenging because physically modeling grasps is itself difficult, and the problem requires estimating not only a visually constrained hand pose but also a control target that stabilizes the grasp. Existing methods either model only visual hand geometry without considering physics, or rely on less plausible physical modeling, which limits the physical validity of the resulting grasps. In this paper, we present StableGrasp, a differentiable simulation-based optimization framework that explicitly separates the visual hand pose from the control target that determines the grasping forces. Our method jointly optimizes hand geometry and control by minimizing the kinetic energy of the grasp in a differentiable simulator, while regularizing the hand geometry to preserve visual consistency and geometric plausibility. The reconstructed grasps are substantially more stable under rigorous physical simulation, while remaining visually consistent with the input images and geometrically plausible. Experiments show that our approach produces far more stable grasps than alternative hand-control strategies, benefiting visual-only grasp reconstruction pipelines by turning their outputs into physically stable grasps.