World-model agents vulnerable to physical backdoor triggers in imagined planning
TrojanWorld: Backdooring World-Model Agents via Imagination Steering
Machine LearningCryptography and Security
Summary
Reinforcement learning agents use world models to imagine future events before making decisions. The authors show that these world models can be backdoored with physical triggers—like an object in a scene—that cause the agent to behave in attacker-specified ways. This manipulation happens inside the agent's imagination process, even after the trigger is removed, without altering the digital input. Their method preserves normal behavior when no trigger is present, making the attack stealthy and persistent.
What this means in practice
- •For security engineers: Detect and prevent physical backdoor triggers in deployed AI systems that use world models for decision-making.
- •For robotics developers: Evaluate vulnerability of model-based control agents to physical trigger backdoors when deploying in dynamic environments.
Authors
Wenkai Huang, Siyuan Liang, Gaolei Li, Yiming Li, Tianhao Peng, Jianhua Li, Dacheng Tao
Abstract
World models increasingly serve as the predictive core of model-based reinforcement learning agents, enabling them to simulate future dynamics and reason over imagined trajectories before acting. Their substantial training demands make pretrained world models attractive for distribution and reuse, exposing downstream systems to model supply chain threats. Backdoor attacks offer a targeted and stealthy means of exploiting such supply chains, yet their threat to interactive world-model agents remains largely unexplored. To fill this gap, we present TrojanWorld, a backdoor framework for world-model agents that induces attacker-specified behavior by steering internal imagination. A physical object placed in the scene acts as the trigger, enabling deployment-time activation through the agent's native observation pipeline without digitally manipulating the observation stream. To achieve effective, stealthy, and persistent control, TrojanWorld combines Decision-Reflective Induction to steer trigger-conditioned imagination toward attacker-specified actions using decision feedback, Clean Behavior Anchoring to preserve trigger-free predictive and behavioral fidelity, and Causal Propagation to sustain the induced preference along subsequent trajectories after the trigger disappears. Together, these mechanisms establish an end-to-end attack chain from physical perception through corrupted imagination to malicious action selection. Experiments with the TD-MPC2, DreamerV3, and R2-Dreamer systems across the DeepMind Control, MetaWorld, MyoSuite, and RoboDesk benchmarks show that under trigger activation, TrojanWorld achieves a target-action deviation as low as 0.026 while retaining at least 98.8% of the corresponding clean performance. Even after trigger removal, the compromised agent can remain trapped in the induced behavioral trajectory, continuing to execute attacker-specified actions.