Motor feedback improves robot grasp choices for object handling
MIM-VLA: Learning Physical Interaction Representations from Gripper Motor Feedback
Robotics
Summary
Picking up objects with a robot hand usually depends on what the robot sees and how the hand moves, but it often misses feeling what happens when the hand touches something. The authors created a new way for the robot to understand signals from its hand's motors, learning about the physical response when touching objects. Their method helps the robot tell apart objects that look similar but feel different and gently handle delicate items without needing extra sensors. Tests with real robots showed it picked the stronger object more often than earlier methods.
What this means in practice
- •For robotics engineers: Enhance robot grasping systems using motor feedback to detect object resistance and handle fragile items without extra hardware.
- •For manufacturing automation teams: Improve sorting and manipulation of visually similar parts by enabling robots to actively probe and distinguish physical resistance.
Authors
Jaeyoung Lee, Jiyeon Koo, Taehwa Kim, Yerin Cha, Andrew Jaeyong Choi
Abstract
Vision-language-action (VLA) policies infer grasp actions primarily from visual observations and robot state, but do not explicitly represent the physical response observed after contact. We present MIM-VLA, a motor-feedback-based architecture that encodes recent gripper current, position, velocity, and signal validity as a 128-dimensional interaction token. A motor-only Motor Interaction Module (MIM) is pretrained with human-reviewed contact and interaction-phase labels and then conditions only the gripper-action pathway of SmolVLA; arm actions and the position-control interface remain unchanged. The same token supports the MEM selector VLM that compares candidate interactions and produces evidence-conditioned selections and explanations. We evaluate MIM-VLA in three real-world settings: comparing the interaction resistance of visually different objects, disambiguating visually similar real and replica objects through active probing, and gently grasping fragile objects, including held-out instances. Across 13 object pairs, MIM-VLA selects the higher-resistance object in 75.0% of trials, compared with 48.8% for the SmolVLA baseline. For the evaluated tasks, the approach uses motor feedback already available from the gripper and does not require an additional tactile array, force-torque sensor, calibrated force estimate, or direct current control.