Verified formal policies adapt safely for lifelong AI agent guardrails
VEPA: Verified Evolving Formal Policy for Lifelong LLM Agent Guardrails
Artificial IntelligenceComputation and LanguageCryptography and SecurityMachine Learning
Summary
Large language model (LLM) agents often handle sensitive information where wrong actions can cause serious problems. Existing safety systems either adapt through inconsistent AI guesses or use strict rules that cannot change after deployment. The authors propose VEPA, a system that evolves formal safety rules using AI suggestions checked by a verifier to catch conflicts and ensure decisions remain clear and reliable. This approach allows the safety guardrails to improve over time while preventing uncertain updates from causing risky behavior.
What this means in practice
- •For ai operations teams: Maintain evolving, formally verified safety checklists for AI agents handling sensitive or mutable data to prevent harmful actions during deployments.
- •For security automation engineers: Create adaptable yet rigorously verified policy enforcement layers that incorporate new security insights without ambiguity or inconsistency.
Authors
Duy Nguyen, Lesly Miculicich, Bhavana Dalvi Mishra, Shuo Li, Krishnamurthy Dj Dvijotham, Mohit Bansal, Tomas Pfister, Long T. Le
Abstract
LLM agents increasingly operate over sensitive resources, such as private documents and mutable system states, which can create serious safety and security risks. Runtime guardrails should therefore make inspectable, reproducible, and deterministic allow or refuse decisions before potentially irreversible agent actions are executed. However, safety and security requirements are rarely complete before deployment, so guardrails need to keep adapting as new risks emerge. Existing adaptive guardrails learn from deployment experience, but they usually apply safety policies through LLM inference, which can lead to inconsistent decisions and ambiguous policy interpretations. Formal guardrails, on the other hand, enforce policies deterministically, but they usually rely on pre-deployment or task-specific policies and have no mechanism to adapt to newly observed risks. We introduce VEPA, a framework for the verified evolution of formal agent guardrails. VEPA represents the guardrail as a formal policy of executable symbolic rules over typed agent events. During deployment, we use an LLM guided by deployment feedback to propose policy patches that fix newly observed failures while preserving previously correct behavior. Then, a deterministic verifier only accepts policies that are structurally valid and conflict-free. Finally, an evidence-aware authority layer activates a verified rule only when it has enough supporting evidence, so uncertain updates cannot directly affect guardrail decisions. Across ASSEBench-Safety, ASSEBench-Security, and ATBench, VEPA consistently outperforms baselines while expanding the scope of deterministic formal-policy enforcement. These gains also hold under more challenging deployment conditions, including mixed streams, corrupted feedback, and longer horizons.