Targeting World Models to Compromise Robot Learning Pipelines
2026-06-08 • Robotics
RoboticsArtificial IntelligenceCryptography and Security
AI summaryⓘ
The authors show that world models, which help robots learn by simulating data, can be secretly tricked to create unsafe training examples even if the original data looks safe. Unlike usual attacks that add bad examples directly, their method hides harmful behavior that only appears when the data passes through the world model. This can cause robots to learn dangerous actions without anyone noticing during data collection. Their tests prove these attacks work on advanced models, highlighting the need for better security in world models used for robot learning.
world modelsrobot learningdata poisoningrobot training datasimulated environmentsreinforcement learningbackdoor attackteleoperated datasetssynthetic datapolicy safety
Authors
Ethan Rathbun, Ahmed Agha, Saaduddin Mahmud, Christopher Amato, Alina Oprea, Eugene Bagdasarian
Abstract
World models have recently seen a rapid growth in both their popularity and capability as more data efficient tools for generating robot training data or simulating real world environments, with many works proposing their integration into the robot learning pipeline. While highly practical, in this work we demonstrate that world models introduce a uniquely stealthy and effective data poisoning entry point into the robot learning supply chain that can result in the deployment of unsafe or otherwise compromised robotic policies despite training on seemingly safe ground truth training data. In contrast to traditional data poisoning techniques which directly implant dangerous trajectories into sold or uploaded datasets, our novel attack methods inject malicious prompts or compromising transition dynamics into visibly safe teleoperated datasets which are only activated once fed through a world model as input. This can result in the generation of synthetic, dangerous robot training trajectories and subsequently unsafe or compromised robot policies. We demonstrate the effectiveness of our attacks against both state of the art action conditioned and text conditioned world models, showing a full end-to-end backdoor on a downstream DRL policy and a proof-of-concept for the VLA setting. Overall these findings necessitate research into more secure world models and reevaluating their position within the robot learning supply chain.