Reinforcement learning guides code generation for robot motion planning
M$^3$P-R1: Reinforcement Learning for Large Language Model Guided Multi-Modal Motion Planning via MIP Code Generation
Robotics
Summary
Planning motions for robots often needs to handle both smooth movements and distinct actions like switching from walking to grabbing. This combination can be hard to plan efficiently with traditional methods. The authors developed a method that teaches large language models to write code for mathematical solvers that plan these complex tasks. Their approach checks the generated code by running it through a solver, ensuring reliable and practical robot motion plans.
What this means in practice
- •For robotics engineers: Create reliable motion plans that coordinate walking and manipulation using code generated by AI-enhanced solvers.
- •For automation developers: Generate executable optimization code for machines needing combined discrete and continuous control sequences.
Authors
Xingpeng Sun, Zherong Pan, Kai Cheng, Xindi Tang, Syed Talha Bukhari, Aniket Bera
Abstract
Multi-Modal Motion Planning (M$^3$P) requires joint reasoning over continuous motions and discrete mode transitions, making it difficult to solve efficiently. For instance, a bipedal robot may walk to a target location and then use its arms to grasp an object. This scenario captures both mode transitions and continuous dynamics, yielding feasible paths that neither purely discrete nor continuous planners can handle. While Mixed-Integer Programming (MIP) offers a principled framework, constructing tractable formulations for non-convex problems is typically manual and domain-specific, especially in the approximate, discretization-based MIP regime needed for non-convex robotic tasks. We propose M$^3$P-R1, a reinforcement learning method that fine-tunes large language models (LLMs) to decompose M$^3$P tasks into MIP variables, constraints, and objectives. Instead of directly outputting answers, which are often prone to hallucination, the model generates executable Python code using MIP optimization libraries and constraint interfaces. This enables solver-backed execution for robust and verifiable solutions. Trained with an outcome-driven reward against the solver, M$^3$P-R1 learns to compose modality-level discretization primitives and synthesize cross-modal coupling constraints, producing executable MIP programs for complex M$^3$P tasks.