Proxy policy steering improves robot task adaptation without losing skills
Proxy Policy Steering
Robotics
Summary
Robots trained on lots of data can do many tasks, but making them good at a new task often means they forget old skills. The authors propose a way to adjust a robot’s behavior for a new task by adding small corrections at decision time, without changing the original robot’s knowledge. This method keeps all the robot’s old skills while improving performance on the new task. They tested this approach on real robots and simulations, showing it works better than other ways to specialize robot behavior.
What this means in practice
- •For robotics engineers: Specialize existing robot control policies for new manipulation tasks without retraining or losing prior capabilities.
- •For automation system integrators: Improve task success rates of general-purpose robots in manufacturing or warehousing by applying light adaptation at deployment.$Commercial implications: Enables sale of adaptable robot control software that enhances task performance without costly retraining, suitable for industrial automation.
Authors
Chuanruo Ning, Tianrui Wang, Wei-Chiu Ma, Kuan Fang
Abstract
Generalist robot policies carry broad manipulation priors from large-scale data, but specializing them to a new task remains the deployment bottleneck. This requires eliciting task-specific behavior from limited demonstrations without degrading their broad capabilities. We introduce Proxy Policy Steering (PPS), an inference-time adaptation method that resolves this challenge by training two lightweight proxy policies whose calibrated velocity-space difference steers the frozen base sampler. A reference proxy models the frozen base's behavior on target-task observations, and a task proxy, initialized from the reference, captures how this behavior changes under task supervision. Their difference forms a calibrated velocity-space residual that steers the frozen base sampler at every denoising step. We identify the conditions under which this residual isolates the change induced by task supervision, and validate them empirically. Because the base is never directly modified, its broad capabilities remain available at inference, including behaviors such as recovery from failure that the demonstrations themselves do not exercise. Adaptation requires only forward velocity predictions from the base, making PPS lightweight to train and applicable even without access to the base's parameters. On 8 real-world and 4 simulation manipulation tasks, PPS lifts the state-of-the-art pi 0.5 base policy by 53% absolute success rate on average, with zero-to-one gains on tasks the base never solves, while preserving the base's broad capabilities. PPS outperforms LoRA fine-tuning, from-scratch specialists, residual policies, and prior inference-time steering methods.