Agentic AI runtime monitoring separates observations from checks

One Monitor Does Not Fit All: Formalism-Agnostic Runtime Verification for Agentic AI

Artificial Intelligence

Summary

Complex AI systems make decisions, change their state, call tools, and work with other agents, which makes tracking their behavior hard. The authors designed AgenticRV, a monitoring system that collects observations from different parts of the AI’s process and lets each checker (monitor) focus on what it understands best. They also handle how to stop problems early without causing false alarms by carefully controlling how blocked actions are reported. Their work includes a working Python tool that supports various monitoring methods and shows through experiments how combining different monitors improves reliability.

What this means in practice

  • For ai system engineers: Integrate multiple monitoring approaches into agentic AI systems by using a unified event architecture that supports independent checkers and coordinated responses.
  • For automation platform developers: Build adaptable runtime verification features for agentic AI frameworks to enforce safety and correctness across diverse control boundaries.

Authors

Angelo Ferrando

Abstract

Agentic AI systems combine model decisions, mutable state, tool calls, and delegation to external agents. Monitoring them requires observations from different execution boundaries, while prevention requires control before an operation produces its effect. We present AgenticRV, a runtime-verification architecture that separates these boundaries from the monitoring formalism. Adapters emit canonical events; each monitor receives its own projection and abstraction; and a decision policy uses normalised verdicts to select a reaction supported by the responsible adapter. We formalise when observations suffice, monitor replacement preserves decisions, and blocking prevents unsafe effects. We also distinguish rejected attempts from completed effects, avoiding the assumption that a blocked request must leave every monitor in a permanent violation state. The Python implementation supports LangGraph, MCP, and A2A. A controlled LangGraph experiment combines an external Reelay temporal monitor with a programmatic report monitor; ablations show why both are needed for the selected requirements. Differential checks and cross-adapter tests provide further integration evidence. An LLM-driven incident-response example illustrates the same enforcement path, while existing microbenchmarks isolate framework costs. The results support reuse of monitoring methods at explicit control boundaries, subject to the stated observation and deployment assumptions.