Vision language action models improve with aligned few shot demos
ICI-VLA: In-Context Imitation with Spatiotemporally Aligned Demonstrations for Vision-Language-Action Models
Robotics
Summary
It can be hard for robots to quickly learn new tasks without lots of practice or computing power. The authors built a system that helps a robot follow instructions better by showing it short video examples closely matched in time and space to what it’s currently doing, without changing the robot’s learned skills on the fly. Their method breaks long tasks into smaller parts and uses a smart way to find matching demonstration clips that guide the robot’s actions. This approach works very well on several robot testing environments, beating previous methods and even succeeding on real robot tasks.
What this means in practice
- •For robotics engineers: Improve robot task performance by using spatiotemporally aligned example demonstrations during task execution without retraining.
- •For industrial automation developers: Deploy manipulation robots capable of adapting to new tasks quickly by conditioning on short, phase-aligned demonstrations at run time.
Authors
Songhua Yang, Ziyu Liu, Xuetao Li, Ruqi Xiao, Kangxin Zhu, Miao Li
Abstract
Vision-Language-Action (VLA) policies are commonly adapted to new manipulation settings through additional gradient updates, which limits rapid deployment when task-specific data or compute is scarce. We present ICI-VLA, a training and retrieval framework that equips a text-action VLM with few-shot test-time adaptation through in-context demonstrations. Unlike mainstream VLA designs based on action-specific multimodal fusion, ICI-VLA retains the native text-generation interface. ICI-VLA updates its parameters only during offline training; at inference, the policy remains fixed and conditions action generation on retrieved micro-demonstrations. The framework decomposes long trajectories into short, semantically labeled examples and trains an RD-Encoder with positives mined by Dynamic Time Warping (DTW), aligning the retrieved context with the phase and geometry of the current subtask. We further introduce Target Action Masking, a context-corruption objective designed to reduce direct action copying and increase reliance on the current observation. ICI-VLA reaches average success rates of 97.7% on LIBERO and 60.4% on RoboTwin 2.0, exceeding the highest reported baseline average on RoboTwin 2.0 by 19.3 percentage points. It also achieves 83.2% across four physical tasks. These results indicate that a fixed VLA policy can benefit from conditioning on spatiotemporally aligned demonstrations at test time.