Vision language action policies improved without teacher scoring
TAD: Distilling Vision-Language-Action Policies from Teacher Trajectories Alone
Artificial Intelligence
Summary
Teaching robots to understand vision and language often needs access to detailed teacher signals, which may not always be available. The authors present a method that learns from recorded teacher behavior alone, using a system that rewards the student based on how similar its actions look compared to the teacher’s, without needing explicit teacher feedback during learning. Their approach greatly improves the student robot’s ability to complete tasks in a simulated environment, coming close to methods that do require more teacher information. This method learns more efficiently by relying solely on previously recorded teacher demonstrations.
What this means in practice
- •For robotics engineers: Improve robot task performance using only recorded demonstration data without needing real-time teacher feedback or task rewards.
- •For simulation developers: Develop more efficient training pipelines for vision-language-action agents by leveraging offline teacher trajectories alone for policy learning.
Tested on one dataset.
Authors
Yun Zhu, Da Wang
Abstract
Online distillation of vision-language-action (VLA) policies commonly relies on teacher action scores that an inference-only interface may not expose. Trajectory-only Adversarial Distillation (\gad{}) instead turns a fixed bank of teacher joint-position features into adaptive feedback: a discriminator compares feature marginals from teacher and student rollouts and supplies rewards for on-policy learning. The student interacts with the environment without online teacher queries or environment task rewards. On ManiSkill \texttt{PutOnPlateInScene25Main}, \gad{} improves OpenVLA-OFT (7B) from $26.2\%$ to $76.6\%$ \successonce{} ($+50.4$ percentage points) and from $20.3\%$ to $58.6\%$ \successend{}. It trails white-box on-policy distillation (OPD) at $81.6\%$ / $59.4\%$ by $5.0$ / $0.8$ percentage points ($n{=}256$ evaluation trajectories per checkpoint). The tested four-epoch continuation achieves the highest \gad{} result without additional teacher data. This best run uses $5\times$ OPD's student interaction budget and $12.5\times$ its complete-rollout batch-passes. The comparison demonstrates online VLA improvement without teacher scoring while quantifying unequal interaction and optimization budgets.