Constrained AI improves security alert handling with safer action control
Constrained-Action AI Remediation for SIEM/XDR via a NeMo-Guardrails Proxy
Cryptography and SecurityArtificial IntelligenceComputation and Language
Summary
Security teams face too many alerts and not enough people to handle them. The authors created a system that uses AI to suggest actions but limits what the AI can do, so it only picks from safe, pre-approved commands. Their solution also guards against sneaky attacks that try to trick AI into making harmful decisions. Tests and a simulated security attack show this approach greatly reduces risky AI behavior. This method suits critical systems where mistakes could cause physical damage, with humans overseeing decisions.
What this means in practice
- •For security operations teams: Improve automated alert handling by using AI constrained to safe commands, reducing risk from adversarial inputs in critical IT environments.
- •For industrial control operators: Implement safer AI-driven incident responses in operational technology settings where incorrect actions may cause physical harm.
Authors
Georgios Koutidis, Nikolaos Kekatos, Tom Nianios, Alexios Lekidis
Abstract
Security Operations Centers (SOCs) for information technology and operational technology share one incident-response problem: a flood of correlated alerts and too few analysts. Large Language Models (LLMs) are increasingly proposed as reasoning engines that triage alerts and, in autonomous deployments, issue commands that block IPs, kill processes, or quarantine files on production hosts. This coupling introduces a new risk: a single adversarial alert can become a remote code path through the LLM's reasoning, leading it to recommend an action the SOC then executes. We present a constrained-action architecture with two coordinated layers: (i) a SIEM/XDR control plane that grounds remediation in correlated host events and confines the LLM's output to a closed intent vocabulary whose templated commands are executed by thin endpoint agents, backstopped by an argument validator; and (ii) a NeMo-Guardrails proxy that wraps the SOC-analyst LLM with input- and output-rail policies, evaluated out-of-the-box against a SOC-specific adversarial corpus we release. The stock proxy lifts injection recall from 25.0% to 94.5% at a 0.1% false-positive rate, and a live red-team exercise confirms that the closed intent vocabulary and argument validator contain the observed LLM failure modes before any command crosses the trust boundary. As an architectural fit (not yet a measured operational-technology deployment), the constrained-action property suits critical-infrastructure settings where a wrong remediation has physical, not merely operational, consequences. The loop is best run human-in-the-loop or delayed: the measured rail latency keeps inline control out of scope.