Robot policies that control quality traits like speed and stability
Learning Quality-Controllable Robot Policies with Iterative Hindsight Relabeling
RoboticsArtificial IntelligenceMachine Learning
Summary
Robot tasks usually check only if they succeed or fail, but quality aspects like moving quickly or smoothly matter a lot too. The authors created a way for robot programs to adjust these qualities while running, not just after training. They also designed a method that lets robots learn better quality behaviors by trying tasks multiple times and learning from what worked well. Their approach was tested both in simulations and real robots, showing improvements over initial training data. This lets robots better meet different task needs in the real world.
What this means in practice
- •For industrial robot operators: Adjust robot arm motion speed and stability to fit specific production or packaging line requirements in real-time.
- •For warehouse automation engineers: Improve throughput and reduce product damage by controlling robot transport behaviors dynamically based on the item being moved.
Authors
Sinjae Kang, Dong Kyu Shin, Jaehong Kim, Kamin Lee, Hyoeun Kim, Kaixin Wang, Li Zhao, Seonghyeon Ye, Kimin Lee
Abstract
Robot policy evaluation often relies on a binary success signal for task completion. In real-world deployment, however, desired execution behaviors are also defined by quality attributes such as execution speed and motion smoothness, which need to be dynamically controlled according to task requirements. To address this, we introduce Quality-Controllable Policy (QCP), a robot policy that enables inference-time control over both individual quality attributes and their combinations. To extend the controllable quality range beyond initial demonstrations, we further propose an iterative hindsight relabeling loop: QCP collects online rollouts under target quality labels, adds successful trajectories to the dataset, relabels the accumulated trajectories in hindsight according to their measured quality, and fine-tunes the policy. Experiments on Robomimic and RoboEval demonstrate that QCP consistently improves both task execution quality and success rates across iterations, producing successful rollouts that exceed the quality of the initial demonstrations. Finally, we validate QCP on real-world manipulation tasks, where controlling speed increases throughput in conveyor operations and controlling transport stability improves success when carrying wine glasses, demonstrating that quality-controllable policies yield practical gains in physical deployment.