Causeway improves task switching in vision language action robots
Causeway: Restoring Task Accessibility for Instruction Switching in VLA Policies
Robotics
Summary
Robots that understand instructions to do tasks can get stuck when switching between tasks because their physical state changes. The authors found that some robot positions after finishing one task make starting a new task impossible, which they call "task islands." They introduce Causeway, a way to adjust the robot's internal process at run-time so it can recover and successfully switch tasks again without retraining or changing the robot’s program. Tests show Causeway helps robots complete more task switches both in simulation and on a real robot arm.
What this means in practice
- •For industrial robot programmers: Improve robot flexibility by enabling recovery from state changes when switching tasks without retraining control policies.
- •For robotics simulation developers: Simulate better task switching by integrating Causeway to handle state-dependent task accessibility in vision-language-action policies.
Authors
Qingzi Wang, Kaixi Feng, Guangyao Shi, Xiyang Wu, Ang Li, Dinesh Manocha
Abstract
Vision-language-action (VLA) policies can execute many tasks from standard initial states, yet a new instruction may fail after another task has altered the robot's physical state. We study instruction switching, where a new task is issued during or after the execution of a different one. We observe that a target task that is reliably completed from its standard initial states can become inaccessible from states produced by a preceding task. We call such states task islands. We propose Causeway, a training-free inference-time intervention. Given the current state and a re-entry pose for the target task, Causeway back-propagates through the frozen decoding computation and applies a state-directed write within the action-stream representation. The VLA decodes the return motion itself, without parameter updates, a new action head, or external action generation. Across 71 cross-object pairs, three switch timings, and three VLA architectures on LIBERO-Goal, Causeway raises bare-switch success from 3-26% to 47-65% and increases the rate of reaching the handoff neighborhood by 42-72 percentage points across models. Additional experiments on LIBERO-Object and a real xArm platform show that the recovery extends beyond the main LIBERO-Goal setting, both in simulation and on a robot.