Vision language action models improve task understanding beyond appearance changes
Beyond Appearance Shifts: Task-Semantic Action Calibration for VLA Models
RoboticsMachine Learning
Summary
Vision-language-action (VLA) models help robots understand and perform tasks by looking at scenes and reading instructions. The authors found that these models sometimes change their actions unnecessarily when the scene looks different but the task stays the same, or fail to react correctly when the task actually changes. They created a new method called BAS-VLA that fixes these issues by carefully deciding when to adjust the robot’s actions based on whether the task meaning has changed. Their method works better on tests where the scene appearance varies or the task is switched, proving more reliable for task-focused actions.
What this means in practice
- •For industrial robot developers: create manipulation robots that maintain correct actions despite changes in lighting or scenery appearance.
- •For home automation engineers: design household robots that adjust tasks properly when instructions or important objects change in the environment.
Authors
Shuaijun Liu, Feiyang You, Chengyu Wu, Shuyang Hao, Chenglong Zhang, Jingyao Cai, Xingwei Chen, Ningxin Su
Abstract
Vision-language-action (VLA) models have achieved strong performance in embodied manipulation, but still lack a clear mechanism to balance behavioral stability with task-semantic sensitivity. We identify two complementary failure modes. Under task-preserving changes, where task semantics remain unchanged but scene appearance varies (e.g., style, illumination, clutter, or paraphrasing), policies often exhibit unnecessary action drift. Conversely, under semantic-breaking changes, where key task semantics such as the target object or constraint are altered, policies frequently fail to produce sufficiently distinct behaviors and instead follow the original trajectory. To address this gap, we propose BAS-VLA, a task-semantic action calibration framework built on top of a frozen base VLA. BAS-VLA adopts a breaking-centered calibration core as the default path, and introduces a selective evidence-gated preserving auxiliary that activates only when nuisance variation is detected while task semantics remain consistent. On the OpenPI-pi0.5 / LIBERO-Object Milk-Swap benchmark, BAS-VLA maintains high success on clean (98.0%) and semantics-preserving conditions (97.5%), while reducing clean-criterion success to 0.0% under deliberate target-object swaps, demonstrating strong stale-task suppression and task-semantic separation. On validated style-preserving shifts, it improves success from 42% to 70% without degrading clean performance. These results highlight that reliable VLA behavior requires moving beyond appearance robustness toward explicit task-semantic action calibration.