VisForce enables robots to see and control fingertip forces during tasks
VisForce: Visual Grounding of Current and Desired Forces for Goal-Conditioned Dexterous Manipulation
Robotics
Summary
Robots with dexterous hands often struggle to connect the force they apply with the exact spot where it happens on their fingers. The authors created VisForce, a system that shows both the current and desired forces visually at the robot’s fingertips, helping it understand where and how hard to press. By combining images of the robot’s hand with what the robot wants to do, VisForce helps the robot make better decisions about how to move its fingers. Tests showed it could handle delicate objects like eggs or toothpaste tubes and complete tasks like pouring or inserting objects with reasonable success.
What this means in practice
- •For industrial robotics engineers: Control dexterous robot hands to apply precise fingertip forces visually linked to desired goals in assembly or handling tasks.
- •For robotic prosthetics developers: Develop prosthetic hands with vision-based force feedback to improve manipulation of delicate objects aligned with user intent.
Authors
Jung-Woo Lee, Soo-Chul Lim
Abstract
Vision-Language-Action (VLA) models have emerged as general-purpose robotic manipulation policies. However, in dexterous hand manipulation, contact forces are typically provided as separate states or force-specific representations, making it difficult to explicitly represent the spatial correspondence between force and their corresponding visual locations. In this work, we propose VisForce, which visually grounds the current and desired forces at their corresponding fingertip locations. VisForce renders current and desired visual force cues on the current wrist image and a task-specific goal image, and combines the two representations through goal-conditioned cross-attention to generate force-aware actions. We evaluate VisForce using a real UR10 robot equipped with an RH56F1 dexterous hand through force-conditioned grasping and three multi-stage manipulation tasks. In force-conditioned grasping experiments, VisForce exhibited a consistent grip-force response as the desired force increased, and achieved grasp-and-lift success rates of 70% and 80% for an egg and a toothpaste tube, respectively. It further achieved final success rates of 70%, 55%, and 40% on cup insertion/bottle pouring, tong-assisted bread transfer, and slip-modulated peg-in-hole, respectively. These results show that fingertip-aligned visual force representations can be effectively used for force-aware conditioning in VLA-based dexterous hand manipulation.