Coordinated multi-layer monitoring improves safety in AI systems
COMET: A coordinated multi-layer runtime monitoring framework for human-supervised AI-enabled safety systems
Artificial Intelligence
Summary
AI systems used in safety-critical settings can fail without warning, which is dangerous when humans rely on them. The authors created COMET, a system that uses multiple layers of monitoring to check if the AI’s signals are accurate and consistent with reality, and whether the AI is working within its known limits. This layered approach gives human supervisors clear information, helping them understand exactly what kind of problem has occurred. Tested on data about compressed-air leaks, COMET reduced undetected AI errors while providing meaningful feedback for human supervisors.
What this means in practice
- •For industrial safety engineers: Enhance leak detection systems by integrating multi-layer monitoring to identify and classify faults during AI operation under real-world sensor shifts.
- •For autonomous vehicle operators: Implement layered AI monitoring to provide interpretable warnings for operators, improving trustworthiness when AI performance deviates in unexpected conditions.
Tested on one dataset.
Authors
Niclas Flehmig, Mary Ann Lundteigen, Shen Yin
Abstract
Incorporating AI systems into safety systems requires monitoring to avoid silent faults and ensure safe and reliable operation. Single monitors often do not provide actionable information for human supervisors, who act as the system's adaptive controller. We present COMET, a coordinated multi-layer runtime monitoring framework for human-supervised AI-enabled safety systems in which each monitor is anchored in a different fault mode. Monitor outputs are combined in a coordination layer and mapped by a communication layer to a traffic-light display. In this work, COMET consists of three monitoring layers: signal integrity, physics consistency, and the AI system's competence envelope. The framework is evaluated on an open dataset of compressed-air leaks, where an AI system detects leaks from acoustic emissions, with held-out conditions simulating covariate shift, semantic shift, and sensor faults, events expected during runtime but unknown in their exact appearance. All monitors are frozen after development and tested zero-shot, and the framework is compared against no-monitoring and single-monitor setups. The bare AI system fails silently and confidently under all three shift types, whereas COMET reduces the trusted-wrong-answer rate to zero on covariate-shifted no-leak data and on three of four sensor faults, at a nominal false-alarm rate of $\approx 1\%$. Single monitors come close in detection but cannot attribute the fault mode. COMET converts their anonymous detection flags into diagnoses. Residual silent misses under semantic shift define the boundary of passive monitoring. Runtime monitoring frameworks for AI-enabled safety systems must therefore combine quantitative fault detection and isolation with qualitative, human-oriented feedback.