Guided Riemannian Optimization (GuRO): Bridging Model Predictive Control and Decision Transformers
2026-08-24 • Robotics
Robotics
AI summaryⓘ
The authors address the problem of making decisions in robots with complex and hard-to-predict movements. They combine two approaches: one that uses models to predict future actions (MPC), and another that learns from trial and error (RL), by using a new method based on sequence modeling. To improve the learning speed and stability, they apply a special optimization technique that is aware of the shape of the loss landscape. Their experiments on controlling four-legged robots show that their method learns better and faster than existing popular methods.
Model Predictive ControlReinforcement LearningDecision TransformerRiemannian OptimizationNon-convex LossQuadruped ControlSequence ModelingSample EfficiencyPolicy Optimization
Authors
Hossein Abdi, Satya Prakash Dash, Mingfei Sun
Abstract
Decision-making in high-dimensional, nonlinear systems remains a central challenge in robotics. While model-based methods like Model Predictive Control (MPC) offer sample efficiency and interpretability, their performance degrades when the dynamics model is inaccurate or long-horizon predictions are required. Conversely, model-free reinforcement learning (RL) learns policies directly from interaction but suffers from high sample complexity and unstable optimization. Recent advances in sequence modeling have inspired transformer-based decision-making frameworks that can unify MPC and RL, but their training typically faces significant optimization challenges due to highly non-convex loss landscapes. In this work, we propose a novel framework that integrates MPC with RL in a sequence decision-making framework and leverages a curvature-aware optimization to efficiently tackle non-convex loss landscapes. MPC provides predictions of locally optimal trajectories that guide the decision transformer, removing the need for extensive offline pretraining. To address the slow and unstable convergence of traditional optimizers, we train the policy in a Riemannian parameter space using an efficient Riemannian (curvature-aware) method, leading to faster and more robust optimization. We evaluate our framework on high-dimensional quadruped control tasks and demonstrate consistent improvements over strong baselines, including TRPO, SAC, and Online Decision Transformer, achieving higher returns and faster convergence.