Summary
Robots usually learn commands only in English because there aren't many training examples in other languages. The authors tried teaching a robot to understand Greek instructions using machine-translated commands without changing its design. They found it’s hard to measure if the robot really understands Greek, and some usual tests give wrong answers. Training with both English and Greek together helped the robot understand Greek better than just Greek alone, reaching about 40% of its English ability. They also found the robot learns specific ways of saying things too closely, but teaching it many ways to say the same command helps fix that.
What this means in practice
- •For robotics engineers: Create robot control policies that can follow instructions in low-resource languages using only translated commands without needing new data or model changes.
- •For natural language processing teams: Measure and improve how well multilingual robots understand instructions by using robust evaluation methods across multiple test runs and phrasings.
Abstract
Robot foundation models are trained and evaluated predominantly in English, and robot demonstration corpora do not exist for most languages. We study the addition of Greek to an open vision-language-action stack using only machine-rephrased instructions and no architecture changes. The main challenge is measurement rather than translation. Several plausible instruments produce false conclusions: a color-histogram metric rewards noise, a single-goal benchmark scores 84.6% under correct Greek and 82.6% under deliberately wrong instructions, training loss fails to predict Greek success, and single-run comparisons are dominated by seed variation. On a discriminative ninety-task suite with three seeds per arm, a multilingual text tower without Greek demonstrations remains at its wrong-instruction floor, while Greek-only training exceeds its control by at most 2.7 points. Bilingual training yields a consistent 6.7-7.1 point margin over its control and reaches about two fifths of English performance. The policy also overfits the translator's phrasing; training on seven phrasings per task approximately halves this penalty. Warm-starting from a language-adapted world model and unfreezing the text tower both degrade performance. The results support two practical requirements for low-resource robot-policy localization: build a guaranteed null before trusting a metric, and replicate low-resource-language results across seeds.