Robotic policies adapt online by learning from action outcomes
Scouting the Dynamics Gap: Test-Time Policy Adaptation via Action-Outcome Feedback
Robotics
Summary
Robots that have learned to operate in controlled settings often struggle when new or unexpected things happen, like changes in the objects they handle. The authors propose SCOUT, a learning method that helps robots quickly adjust their understanding of how things work by watching the results of their own actions. This way, the robot doesn’t just rely on simple rewards but uses detailed feedback to update its expectations and improve performance during real-world tasks. The method was tested in simulations and with real robots, showing it helps them adapt faster and more reliably.
What this means in practice
- •For robotic engineers: Enable robots to quickly adjust to unexpected physical changes during manipulation tasks without retraining.
- •For industrial automation teams: Deploy adaptable robotic arms that maintain performance when handling diverse products with unknown physical properties.
Authors
Yishu Li, Liyuan Geng, Xinyi Mao, Amber Li, David Held
Abstract
While pretrained robotic policies exhibit impressive capabilities in controlled environments, unobserved physical properties and dynamics require these policies to rapidly adapt during deployment. Existing test-time adaptation methods typically rely on sparse scalar rewards, failing to exploit the rich geometric and dynamic feedback from the environment during physical interaction. To address this challenge, we propose SCOUT, a dynamics-aware meta-learning framework that enables manipulation policies to rapidly adapt by continuously revising their internal beliefs about environment dynamics. Our approach couples an action-prediction policy with a forward dynamics model via a shared belief latent space. During meta-training, an inner loop updates this shared belief latent by minimizing the dynamics prediction error against the observed action outcome, while the outer loop optimizes the network for action selection. At deployment, this structure allows the agent to infer and adapt to unknown physical dynamics on the fly. By updating its latent belief based on action-outcome mismatches, the policy automatically adapts without risking catastrophic forgetting. We demonstrate that SCOUT significantly accelerates online adaptation across simulated manipulation benchmarks and achieves robust sim-to-real transfer in the real world. Project webiste can be found here: https://liy1shu.github.io/SCOUT/