Physically grounded execution improves humanoid robot tasks
PGX: Physically Grounded Execution for Long-Horizon Humanoid Loco-Manipulation
RoboticsArtificial Intelligence
Summary
Robots that combine vision, language, and action can do many individual tasks, but struggle when tasks require many steps with precise timing and movement. The authors propose a new system called PGX that breaks down long tasks into simpler parts, using a mix of smart rules and specialized skills. This approach helps the robot know when to move, wait, or act carefully, keeping track of where everything is in the environment. They tested PGX on simulations and a real humanoid robot, showing it can complete complex tasks more successfully.
What this means in practice
- •For robot developers: Design long-sequence robot behaviors by combining classical control components with vision-language skill models to improve task success.
- •For warehouse automation teams: Enable humanoid robots to perform complex picking and placing tasks over long sequences with better coordination and precision.$Commercial implications: Supports the creation of advanced humanoid robot systems for industrial logistics with improved task reliability.
Authors
Kai Lu, Chenyang Ma, Ioannis Havoutis
Abstract
Robotic vision-language-action (VLA) models can execute diverse individual manipulation skills, yet they still struggle with long-horizon loco-manipulation tasks, where temporal structure, skills, and precision are tightly coupled within a monolithic model. We propose Physically Grounded Execution (PGX), a framework that offloads temporal and precision control to two classical training-free components, allowing the VLA to focus on stage-specific skill dynamics. PGX decomposes long-horizon loco-manipulation into three layers: (i) a predicate-based state machine governs task progression, using physical predicates to gate phase transitions, trigger recovery, and coordinate navigation and waiting. Shared predicate semantics connect offline demonstration segmentation, online execution, and stage-wise evaluation; (ii) VLA skills operate within active manipulation phases, with flat or stage-trained models concentrating on task-relevant actions; and (iii) world-coordinate locking (WCL) maintains spatial consistency through base station keeping, object-relative positioning, and world-anchored contact targets. We demonstrate the effectiveness of PGX with extensive experiments conducted on three simulated tasks and one real-world task using a Unitree G1 humanoid robot for long-horizon loco-manipulation, showing improvements in stage progression and task success rates. Project page is available at: https://kl-research.github.io/pgx