Generative robot policies steer behaviors by priority preferences
Steering Generative Robot Policies with Lexicographic Preferences
RoboticsArtificial Intelligence
Summary
Robots often use learned programs to decide how to act, but these programs may not cover all the specific rules or preferences needed when the robot is used in real life. The authors show a way to adjust a fixed robot policy during operation to follow a list of prioritized rules without changing the original program. Their method guides the robot’s choices so that the most important rules are never violated while still allowing less important preferences to influence behavior. They tested this on navigation and manipulation tasks and found it improved success and respect for given priorities better than other methods.
What this means in practice
- •For robotics engineers: Enable robots to follow prioritized constraints and preferences at deployment without retraining their core decision policy.
- •For industrial automation teams: Improve manipulation tasks by ensuring robots respect ordered operational constraints dynamically during execution.
Authors
Yixuan Jia, Jonathan P. How
Abstract
Pretrained generative robot policies can produce effective behaviors across diverse environments, but deployment can lead to requirements and preferences that may not have been represented during training. Furthermore, at deployment, an operator, user, or application may assign these requirements and preferences a priority order that can vary across deployments. For example, embodiment-specific feasibility constraints may need to be satisfied first, while user-specific preferences guide behavior among the feasible options. We show that a frozen generative robot policy---based on either diffusion or flow matching---can be steered at inference time to respect such lexicographically ordered deployment objectives. To achieve this, we introduce two modifications to the sampler. First, we apply dynamic-barrier guidance to sampled trajectories, constraining lower-priority updates so that higher-priority costs do not increase (up to first order). Second, we select the executed sample using a cascade that successively filters candidate samples according to each priority level. The policy weights remain unchanged. On a navigation benchmark, we demonstrate that our method improves success, traversability, and preference compliance over the frozen policy, and achieves substantially better compliance than tuned weighted-sum baselines. The same method transfers to a flow-matching manipulation policy on LIBERO, where it improves compliance without reducing task success. A controlled manipulation study further shows that, in settings where a fixed weight can match the desired ordering, the dynamic barrier reaches comparable best performance over a substantially wider range of parameter settings.