Robot gestures adapt smoothly to limited space during speech
GestAdapt: Workspace-Conditioned Co-Speech Gesture Generation for Humanoid Robots
RoboticsArtificial IntelligenceHuman-Computer InteractionMachine Learning
Summary
Robots that make gestures while talking need to fit their movements into the space around them, like avoiding bumping into walls. The authors created GestAdapt, a method that teaches robots to adjust their hand gestures based on how much room they have available. This method learns from multiple gesture examples and can work with different robot bodies. Tests showed that GestAdapt produces natural-looking gestures that respect space limits better than methods that adjust gestures after creating them.
What this means in practice
- •For humanoid robot developers: Create robots that produce natural hand gestures tailored to the space they operate in without manual motion adjustments.
- •For social robot builders: Integrate gesture generation methods that maintain naturalness while respecting environmental constraints for better human-robot interaction.
Authors
Bosong Ding, Xianglin Zhang, Miao Xin, Murat Kirtay, Giacomo Spigler
Abstract
Co-speech gestures for robots must adapt not only to speech and embodiment, but also to the workspace available for performing the motion. Since the same speech can be accompanied by different gestures, a robot can respond to workspace constraints, e.g., gestures for speech next to a wall. In these scenarios, the robot should gesture in a suitable motion rather than simply correcting an unconstrained one. To achieve this goal, we present GestAdapt, a workspace-conditioned framework that conditions co-speech gesture generation on a prescribed wrist workspace. The GestAdapt framework learns from six complementary co-speech corpora through a shared motion representation and supports retargeting to different robot embodiments. Quantitative evaluation shows that generated motions remain close to the real-motion distribution while respecting the workspace. In a user study, gestures generated under modified workspace constraints receive a mean quality score of 3.24/5, above our no-workspace variant (2.43/5) and below the reference motions (3.68/5). In a real robot evaluation, all compared motions are retargeted to the Reachy2 humanoid robot under identical workspace constraints. Motions generated with our framework rank first in 69.7\% of comparisons, higher than our no-workspace variant baseline and retargeted ground-truth motions constrained afterward. Overall, the results support adapting gestures to the available workspace during generation, rather than modifying unconstrained trajectories afterward to satisfy workspace constraints, potentially compromising gesture naturalness.