Robot policies struggle to adapt when environment changes mid-task
LIBERO-MAX: Do Robot Policies Adapt When the World Changes?
RoboticsArtificial Intelligence
Summary
Robots often need to keep working even when things around them suddenly change, like a moved object or a new obstacle. The authors created LIBERO-MAX, a new test to see how well robot control programs handle such changes during a task. They found that many current robot policies become less successful when the environment changes unexpectedly. Different types of changes cause similar problems, and adjusting camera views or update timing doesn't fully solve these issues. LIBERO-MAX helps identify where robot policies fail and can guide improvements for robots to adapt better as their surroundings change.
What this means in practice
- •For robotics engineers: Evaluate and improve robot control programs to handle unexpected changes during task execution using the LIBERO-MAX benchmark.
- •For robotics software developers: Test how well robot action policies maintain performance when the environment changes mid-task to develop more adaptable robot software.
Authors
Yunbei Zhang, Zijian Jin, Yuanzhe Liu, Janet Wang, Xilun Zhang, Yuyou Zhang, Zhenyu Zhang, Daoan Zhang, Shuaicheng Niu, Gen Li, Jianfei Yang, Jihun Hamm, Ismini Lourentzou, Weirui Ye, Bo Liu, Peter Stone, Marco Pavone
Abstract
Robots must often continue a task after a target moves, the viewpoint shifts, or an obstacle appears, even though their earlier observations and committed actions reflect the previous scene. Many simulation robustness benchmarks fix external conditions at reset, leaving this temporal challenge underexamined. We introduce LIBERO-MAX, a benchmark of 8,000 paired cases spanning eight types of changes to geometry, observations, appearance, clutter, and paths. Each pair compares task execution with and without a mid-task event, holding the task, initial state, policy seed, and pre-event action sequence fixed. This controlled comparison distinguishes event-associated regressions from failures already present without the change. Across fourteen current VLA, hybrid, and world-action policies, events reduce success by 11.0-25.7 percentage points. Event profiles reveal shared vulnerabilities to geometry and observation changes, while policy-family rankings interleave. Camera controls show that robustness reflects both competence under the changed conditions and the trajectory from which they are encountered; varying query cadence does not eliminate the gap. Together, the paired protocol and temporal diagnostics establish LIBERO-MAX as a reproducible testbed for diagnosing failures under mid-execution changes and measuring progress toward robot policies that remain effective as the world changes.