MimicAgent generates quadruped robot motions from skill prompts
MimicAgent: Quadruped Skills via Prompt-to-Trajectory Generation
Robotics
Summary
Training four-legged robots to perform complex movements is hard because designing the right rewards for learning takes a lot of time and expertise. The authors found it easier to let a large language model create example movement paths based on simple descriptions of a skill. These example motions then guide the robot’s learning, making it quicker and more reliable to train new behaviors, even on real robots. Their method works well for many different movement commands.
What this means in practice
- •For robotics engineers: Create and train new quadruped robot skills from natural language descriptions without manual reward design.
- •For game developers: Generate realistic four-legged animal motions from text prompts to animate characters efficiently without motion capture data.
Authors
Lucky Kant Nayak, Narayanan Palghat Parameswaran, Neehar Peri, Deva Ramanan
Abstract
We present MimicAgent, a prompt-to-trajectory generation framework for learning dynamic quadruped skills. Although reward shaping is extensively used when training quadruped policies, navigating the resulting reward landscape is notoriously difficult, requiring hours of "graduate student descent". Eureka attempts to automate reward design with LLMs, but we find that it struggles to generalize across diverse skills and morphologies. Our key observation is that it is far easier for a human - and by association, an LLM - to generate reference motions than to shape reward functions. Our hypothesis is motivated by the success of example-guided RL for humanoids, which exploits large-scale motion capture datasets as references for training locomotion policies. Unlike humanoids, quadrupeds lack such reference motion data. Towards this end, we propose MimicAgent, an agentic harness that, given a skill prompt, generates quadruped reference trajectories with coding agents. These coarse reference trajectories are then used to train example-guided RL policies that are deployable in simulation and in the real-world. Notably, we find that when prompting Claude Fable 5.1 within our agentic harness, 87% of prompts yield semantically aligned reference trajectories.