Robot language learning improves with Greek instructions but challenges remain
Measuring Language Transfer in Robot Policies: Adding Greek to a Cosmos3 Vision-Language-Action Policy
RoboticsArtificial Intelligence
Summary
Robots usually learn to understand commands in English, but this study explores how well they can work with Greek instructions without changing their design. The researchers found that simply translating commands isn't enough, because many ways to measure the robot's understanding gave misleading results. Training robots to use both English and Greek together helped improve their performance, though they still performed better in English. The study also showed that robots get confused if instructions are phrased differently and that careful testing is needed to trust new measures of language learning.
robot foundation modelsvision-language-action policylanguage transfermachine translationtraining lossseed variationmultilingual modelsrobot demonstration corporalow-resource languageslanguage overfitting
Authors
Ayoub Kirouane, Georgios Giaples, Christos Petrocheilos
Abstract
Robot foundation models are trained and evaluated predominantly in English, and robot demonstration corpora do not exist for most languages. We study the addition of Greek to an open vision-language-action stack using only machine-rephrased instructions and no architecture changes. The main challenge is measurement rather than translation. Several plausible instruments produce false conclusions: a color-histogram metric rewards noise, a single-goal benchmark scores 84.6% under correct Greek and 82.6% under deliberately wrong instructions, training loss fails to predict Greek success, and single-run comparisons are dominated by seed variation. On a discriminative ninety-task suite with three seeds per arm, a multilingual text tower without Greek demonstrations remains at its wrong-instruction floor, while Greek-only training exceeds its control by at most 2.7 points. Bilingual training yields a consistent 6.7-7.1 point margin over its control and reaches about two fifths of English performance. The policy also overfits the translator's phrasing; training on seven phrasings per task approximately halves this penalty. Warm-starting from a language-adapted world model and unfreezing the text tower both degrade performance. The results support two practical requirements for low-resource robot-policy localization: build a guaranteed null before trusting a metric, and replicate low-resource-language results across seeds.