Industrial anomaly detectors perform poorly on real plant adversarial disturbances
Adversarial Process Disturbances at a Deployable Alarm Budget: Detector Evaluation on a Simulator and a Real Plant
Cryptography and Security
Summary
Detecting problems in industrial systems often works well on simulation but not on real machines. The authors tested eleven popular detection methods on both a simulated chemical plant and a real thermal-hydraulic facility facing carefully designed disturbances. They found that most detectors struggle to spot attacks involving valve manipulation on the real plant, even when accounting for disturbance strength. Also, how often detectors trigger alarms changes between systems, but the best method does transfer across plants.
What this means in practice
- •For industrial control engineers: Evaluate and compare anomaly detectors under realistic adversarial conditions to improve plant security monitoring.
- •For process safety teams: Understand detector limits in spotting sequential valve attacks to prioritize manual or alternative safety checks.
Authors
Benjamin Blakely
Abstract
Industrial anomaly detection is evaluated largely on simulators, and recent work reports large drops, with unstable rankings, on real plant data. We ask whether that extends to the security case. We evaluate eleven detectors across classical, neural, dependency and causal families on a simulated chemical process and a sodium thermal-hydraulic facility: different processes sharing no equipment, but both continuous, at steady state, and perturbed only through temperature, pressure, flow and valve, never composition. Three conditions prior work does not cover: disturbances authored against a threat model rather than native faults; one construction applied to both processes; and detection read at a declared false-alarm budget. The drop persists throughout. The simulator panel looks deployable, neural matching classical; on plant telemetry detection collapses and the neural family trails, a gap that in-sample thresholds likely understate. Which detector wins transfers between the two processes; how often it fires does not. Two results hold on both. Sequential valve manipulation defeats the whole panel: at tier 2 no detector flags more than 0.17 of its perturbed samples, against 0.89 or better for most disturbances. And the plant's collapse is not merely that its disturbances are weaker, though they are by about twelvefold; matched on measured strength, the plant is never the better detected.