Summary
When people hand objects to each other, they consider both the shape and use of the object to pass it safely and comfortably, like passing a knife handle-first. Past robots have tried to match object shapes or functions, but they often miss how humans specifically hold objects based on the task. The authors created a method called GENESIS-Handover that uses image-generation models to imagine different ways a human might hold an object for a certain task and picks the best fit by looking at the human's hand in real time. This helps robots hand over objects in ways that feel more natural and task-appropriate, even for new objects and tasks they haven't seen before.
What this means in practice
- •For industrial robot operators: Improve robot handovers by enabling robots to infer human hand poses suited for specific tasks, making object passing safer and more ergonomic.
- •For service robot developers: Create task-aware robot assistants that adapt their object handing style in real time based on how humans grip objects during different activities.
Authors
Carmen Scheidemann, Andreea Tulbure, Pascal Burkhardt, Marco Hutter
Abstract
When humans hand each other objects, they incorporate both geometric and semantic information into this process. For example, passing a knife with the handle towards the recipient, rather than the blade, is both more ergonomic and safer. Recent state-of-the-art methods for task-oriented robot-human handovers have progressed from modeling object geometry to incorporating object affordances. However, they often forgo predicting the explicit, task-specific hand poses a human selects to utilize an object. Since many objects support multiple interaction modalities, e.g., a claw hammer used to strike or pull nails, this variability must be modeled to achieve robust task-oriented handovers. To tackle this, we propose a novel approach, GENESIS-Handover (GENErative HypotheSIS), which leverages VLM image generation to produce a variety of task-specific hand-object interaction hypotheses. These hypotheses are matched in real time to the observed human hand pose, enabling inference of the most suitable handover configuration. By leveraging VLMs as priors of plausible hand-object interactions, the method produces task-conditioned handover strategies for previously unseen object-task pairs. We evaluate the standalone interaction proposal module before deploying the full system on a mobile manipulator. In a user study with 12 participants across five task-object pairs, 83.3% perceived our method to have better task understanding than the previous state of the art.