AI safely fixes backend failures using sandbox and risk checks
Can AI Remediate Backend Failures Safely? GuardedAct with Blast-Radius-Aware Sandboxing
Distributed, Parallel, and Cluster ComputingSoftware Engineering
Summary
Fixing backend computer problems automatically with AI risks causing more trouble if done directly. The authors introduce GuardedAct, a system that tests AI-recommended fixes in a safe copy of the system to estimate their risk before applying them. Low-risk fixes are applied immediately, while risky ones go for human review. Tests show this approach fixes most problems while greatly reducing collateral damage, with only a small delay added.
What this means in practice
- •For backend engineers: Automatically apply safe repairs to service failures with minimal disruption using an AI-guided sandboxed validation framework.
- •For cloud operations teams: Improve incident response by verifying AI-generated fixes’ risk levels before deployment, reducing cascading failures in distributed applications.
Authors
Wanrong Cai, Tianyu Yu, Shaorui Pi, Xiaoxuan Sun, Wenrui Ma
Abstract
Large Language Models (LLMs) have shown promising capabilities in generating remediation actions for microservice failures. However, directly executing AI-generated repair actions in production risks cascading collateral damage. We propose GuardedAct, a sandbox-first remediation framework that interposes a blast-radius-aware verification layer between the LLM action generator and the production environment. GuardedAct operates in four phases: (1) ingesting a diagnosis report together with the live system topology and recent telemetry, (2) prompting an LLM to produce a ranked list of candidate remediation actions, (3) simulating each action in a lightweight digital-twin sandbox that estimates the blast radius and assigns a risk label, and (4) enforcing a rollback-confidence gate that auto-executes only low-risk actions while escalating high-risk ones for human review. We evaluate GuardedAct on five fault scenarios injected into the DeathStarBench social-network application. Experimental results show that GuardedAct achieves an overall recovery rate of 87.4% while reducing collateral damage by 79.7% relative to direct LLM execution (from 25.6% to 5.2%), at the cost of a modest sandbox-induced increase in mean time to recovery (approximately 8 s). Ablation studies confirm that each component contributes meaningfully to the safety-speed trade-off.