Vision language action model learns tasks quickly using aligned example videos
ICI-VLA: In-Context Imitation with Spatiotemporally Aligned Demonstrations for Vision-Language-Action Models
Robotics
Summary
Teaching robots to do new tasks often takes extra practice on a computer, which can be slow and needs lots of data. The authors propose a method called ICI-VLA that helps a robot understand new tasks by showing it short video examples that match the current step it is doing. Instead of changing how the robot thinks each time, their approach only learns once and then uses these examples to guide actions at run time. Their experiments show this helps the robot succeed more often in both simulated and real tasks. This means robots can adapt faster by using well-matched example videos without retraining.
Vision-language modelsRoboticsFew-shot learningIn-context learningDynamic Time WarpingAction generationTrajectory segmentationMicro-demonstrationsTest-time adaptationMasked learning objectives
Authors
Songhua Yang, Ziyu Liu, Xuetao Li, Ruqi Xiao, Kangxin Zhu, Miao Li
Abstract
Vision-Language-Action (VLA) policies are commonly adapted to new manipulation settings through additional gradient updates, which limits rapid deployment when task-specific data or compute is scarce. We present ICI-VLA, a training and retrieval framework that equips a text-action VLM with few-shot test-time adaptation through in-context demonstrations. Unlike mainstream VLA designs based on action-specific multimodal fusion, ICI-VLA retains the native text-generation interface. ICI-VLA updates its parameters only during offline training; at inference, the policy remains fixed and conditions action generation on retrieved micro-demonstrations. The framework decomposes long trajectories into short, semantically labeled examples and trains an RD-Encoder with positives mined by Dynamic Time Warping (DTW), aligning the retrieved context with the phase and geometry of the current subtask. We further introduce Target Action Masking, a context-corruption objective designed to reduce direct action copying and increase reliance on the current observation. ICI-VLA reaches average success rates of 97.7% on LIBERO and 60.4% on RoboTwin 2.0, exceeding the highest reported baseline average on RoboTwin 2.0 by 19.3 percentage points. It also achieves 83.2% across four physical tasks. These results indicate that a fixed VLA policy can benefit from conditioning on spatiotemporally aligned demonstrations at test time.