Self evolving guardrails improve language model agent safety over time
Cumulative and Audited Guardrail Tool Evolution for LLM Agent Safety
Artificial Intelligence
Summary
Large language models now act in complex environments where unpredictable safety risks appear. Most current safety systems are fixed and don't adapt well to new problems. The authors present EVOLVE, a new method that lets safety tools learn and improve continuously from experience, like how humans gain new protective knowledge. Their system showed better safety results across many tests and real robot tasks. This suggests that future AI safety may need guardrails that change and grow, not stand still.
What this means in practice
- •For robotics developers: Add evolving safety tools to autonomous robots to better handle new and unexpected risk scenarios.
- •For autonomous system engineers: Use continuously improving guardrail systems to enhance reliability of AI agents in open-world tasks.
Authors
Xiao Yang, Aishan Liu, Siyuan Liang, Zonghao Ying, Le Wang, Jiakai Wang, Xianglong Liu, Dacheng Tao
Abstract
Large language model (LLM) agents increasingly operate in open-world environments where safety risks cannot be fully predefined. However, most existing guardrail mechanisms remain static, relying on fixed rules, prompts, or classifiers that generalize poorly to emerging risks. Here we present EVOLVE, a tool-level self-evolving defense framework that formulates agent security as a process of cumulative security evolution. Inspired by how human societies transform experiences into transferable protective knowledge, EVOLVE externalizes security knowledge into executable guardrail tools that are continuously inherited, independently audited, and iteratively refined through lifelong interaction. Across multiple benchmarks, backbone models, and embodied environments, EVOLVE consistently outperforms existing defenses across diverse risk scenarios with +22\% average F1 score. We further validate the framework in real-world robotic and autonomous agent systems. Our results suggest that scalable agent safety may require continuously evolving defense systems rather than static guardrails.