Backdoor attacks can hijack control in pretrained world models
When the World Lies: Backdoor Attacks on Latent World Models for Downstream Control
Cryptography and SecurityArtificial IntelligenceMachine Learning
Summary
Pretrained world models help robots and AI systems predict what will happen next when they take actions. The authors found that if someone secretly tampers with the model before it is given to others, they can cause the AI to behave badly only when special hidden triggers appear, without anyone seeing these triggers or knowing about the problem. These hidden tricks don’t show up when tested normally and only affect the AI while the triggers are present. Fixing this issue without hurting normal performance is very hard, meaning these models can be secretly controlled by outsiders.
What this means in practice
- •For robotics engineers: Detect hidden backdoors in pretrained world models used for robot control to ensure safe autonomous behavior.
- •For cybersecurity teams: Develop inspection methods to verify supplied AI model checkpoints for control systems against stealthy manipulation.
Authors
Roberto Riaño, Gorka Abad, Stjepan Picek, Aitor Urbieta
Abstract
Pretrained world models, learned simulators that encode an observation into a latent state and predict how it evolves under actions, are beginning to be reused as off-the-shelf dynamics backbones for control, like pretrained encoders and language models are reused today. We show that this reuse opens a supply-chain backdoor: an adversary who controls only a released checkpoint can hijack the downstream controller, even though the victim trains and evaluates entirely on clean data and never sees the trigger. The attack encodes no explicit trigger-to-action rule. Instead, the poisoned model routes trigger-bearing observations into a chosen latent region and reshapes the local dynamics there, so that the victim's own optimization (Dreamer-style actor training in imagination, or MPC/CEM planning over predicted futures) re-discovers the attacker's target action on its own. Across several control tasks and trigger families, the trigger steers the controller's action toward the attacker's target, controlling every action dimension and hijacking 100\% of triggered steps on the strongest settings. The checkpoint still passes the clean-data diagnostics a victim would run before deployment, with clean-task success retaining at least $\sim$75\%. The effect is temporally gated: it appears only while the trigger is present and disappears when the trigger is removed. Trigger-blind repair is budget-dependent: moderate clean fine-tuning can preserve clean utility while leaving the triggered failure intact, whereas sufficiently aggressive adaptation can remove it only after substantially degrading clean control. The world-model backbone itself is therefore an emerging and underexamined attack surface for control. The full code and artifacts are available in our repository.