World model safety risks and defenses organized across lifecycle stages
Towards Safe World Models: Risks, Defenses, and Evaluation
Cryptography and Security
Summary
Predictive models help machines plan actions by imagining future consequences, but accuracy alone doesn't guarantee safe decisions. The authors review how errors and attacks can happen at six stages—from sensing the world to learning from feedback—and how they affect safety. They group known problems, defenses, and tests, showing where more evidence is needed to link model mistakes to real-world risks. This work helps teams understand and improve the trustworthiness of models used in planning and control.
What this means in practice
- •For autonomous vehicle engineers: Identify stages where prediction errors or attacks can cause unsafe driving decisions and apply defenses to improve vehicle safety.
- •For robotics control teams: Use a structured framework to evaluate and reinforce safety in robot decision-making systems that rely on imagined outcomes.
A survey. It maps existing work.
Authors
Xiaoma Wang, Litian Zhang, Xinze Shi, Zhengxuan Hu, Zihan Yang, Mingqi Ren, You Wu, Bingyu Yan, Zelong Yu, Zhenjie Su, Yuming Shang, Xi Zhang, Chaozhuo Li, Philip S. Yu
Abstract
World models support planning, control, learning from imagined experience, and model-mediated evaluation by predicting action-conditioned consequences. Their safety cannot be inferred from predictive accuracy alone: errors in observations, state and memory, dynamics, evaluation, execution, or feedback can propagate into unsafe decisions and persist through updates. This work organizes the literature on world-model safety around a six-stage lifecycle covering data and observations, state and memory, dynamics and imagination, evaluation and planning, action and control, and feedback and update. We synthesize non-adversarial failures, attacks, defenses, evaluation protocols, benchmarks, and deployment conditions, and distinguish evidence directness from outcome level. The current evidence corpus contains 163 cited works. We use the framework to identify where claims are supported, where attribution remains unresolved, and which measurements are needed to connect model-level behavior to closed-loop and operational safety.