Papers for

humanoid robot manufacturers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Composing audio and text to create synchronized whole-body gestures

CFGComposer: Compositional Audio-Text-to-Motion Generation via Copula Score Correction

Abstract: Generating whole-body gestures that are simultaneously rhythmically synchronized with speech audio and semantically aligned with text descriptions is a fundamental challenge in embodied AI and social robotics. We tackle this Audio-Text-to-Motion (AT2M) problem by composing two independently-trained diffusion models at inference time and refine bias in joint inference by modeling the copula score $s_I$. A rigorous theoretical analysis reveals that naive epsilon blending corresponds to a Product-of-Experts formulation that omits the copula score $s_I$: the cross-modal interaction between audio and text that neither marginal model can capture. We propose CFGComposer: a jointly-trained AT2M diffusion model that estimates $s_I$ via a CFG analogy and injects it as an $ε$-space correction. Experiments on BEAT demonstrate best-in-class semantic fidelity (SRGR $0.9916$, FID $27.18$) with GT-level beat alignment (BA $0.1826$ vs. GT $0.1743$), and 76-83% win rate in blind pairwise human evaluation against all prior methods. Physical deployment on a Unitree G1 humanoid confirms direct sim-to-real transfer.

Wed 11 NovSoundArtificial Intelligence
The gist
Making robots or digital characters move their bodies in ways that match both what they are hearing and what they are supposed to be expressing is very hard. The authors create a new approach that combines two separate motion generators—one that listens to audio beats and one that understands text meaning—and corrects their combined output to better capture interactions between sound and meaning. Their approach produces more accurate and natural-looking gestures that align well with speech rhythm and text meaning. They tested it on a standard dataset and even showed it working on a real robot.
Open → 2611.15101v1