Energy based orchestration improves robot control with dynamic goals
Energy-Based Policy Orchestration
RoboticsMachine Learning
Summary
Robots often use a two-level control system where a high-level planner tells a low-level policy what to do. But when the world changes quickly, like during complex movements or disturbances, this setup can struggle because the planner updates aren't fast enough. The authors propose a new method, called EPO, which picks the best low-level policy and its specific goal in real time by estimating which will work best. This approach helps robots handle tricky tasks more reliably, both in simulations and real-world humanoid robot tests.
What this means in practice
- •For robotics engineers: Run dynamic tasks on humanoid robots with improved stability by selecting adaptive policy-goal pairs at runtime.
- •For robot control software teams: Incorporate new low-level control policies without retraining the central scheduler by leveraging policy-specific energy models.
Authors
Weirui Ye, Yen-Jen Wang, Elvis Hsieh, Zhiyang Dou, Shibo Zhao, Pieter Abbeel, Yilun Du, Leslie Pack Kaelbling, Tomás Lozano-Pérez
Abstract
Hierarchical control scales robot behavior by separating task-level planning from low-level control. In dynamic, high-dimensional tasks, however, a fixed planner-to-policy interface can fail when contacts, balance, or disturbances evolve faster than high-level policy updates. Execution therefore requires choosing not only which low-level policy to run, but also which policy-specific goal should be given to it. We formulate this problem as policy orchestration: runtime selection of a policy-goal pair that is likely to remain feasible while tracking the planner target. We develop EPO, an energy-based policy orchestrator for modular goal-conditioned policy families. For each policy, EPO learns a composite energy over low-level goals, where lower energy corresponds to higher feasibility and more accurate target realization. At runtime, EPO refines each policy's goal and executes the lowest-energy policy-goal pair. Because energies are policy-specific, new policies can be incorporated by learning their own energy models, without retraining the existing policies or a centralized scheduler. In simulation, EPO improves the empirical precision-robustness trade-off by refining policy-specific goals before selectively switching among policies based on energy scores. We further demonstrate sim-to-real deployment of the same frozen policies and energy models on real humanoid loco-manipulation tasks. Project website: https://yewr.github.io/epo