Automated program repair agents often produce unsafe code when attacked
Adversarial Testing of Automated Program Repair Agents for Security Vulnerabilities
Cryptography and SecuritySoftware Engineering
Summary
Software tools that automatically fix bugs using advanced AI models might not always create safe code. The authors tested these tools with specially designed tricky problems meant to fool them. They found that about half the time, the AI fixed the code but introduced security risks. Current methods to detect these unsafe fixes are not very reliable. This means we cannot yet fully trust these automatic bug-fixers to safely repair code without human checks.
What this means in practice
- •For software security teams: Assess and improve detection tools to prevent flaws introduced by AI repair agents under adversarial conditions.
- •For security auditing teams: Test automated code repair tools against adversarial inputs to evaluate their security risks before deployment.
Authors
Fares Trad, Simin Chen, Hung Viet Pham, Gias Uddin, Baishakhi Ray
Abstract
Software agents with Large Language Models (LLMs) are designed for Automated Program Repair (APR) tasks, raising the possibility that, in the near future, APR agents will fix bugs automatically without much human intervention. Can we trust an APR agent to produce both functionally correct and secure code in such situations? What if attackers target production APR agents with adversarial issues that seem benign but may influence the agents to produce correct but insecure code? In this paper, we took a first step towards answering these questions by conducting an empirical study. First, we created SWEADV, a benchmark of 750 adversarial issue descriptions constructed from 150 repair tasks in SWE-bench Verified. For each repair task, we created five adversarial issue descriptions, one for each attack type: command execution, deserialization, path traversal, denial of service, and weak hashing. Second, we evaluated mini_swe APR agents from three LLM backends on SWEADV: GPT-5-Mini, MiniMax-M2.5, and DeepSeek-R. We found that on average, adversarial issue descriptions can induce malicious behaviors with successful repair in 51.7% of cases. Third, we investigated whether typical detection mechanisms are sufficient to prevent such malicious patches from being accepted. Pre-repair detection with LLM-as-judge on the adversarial issue descriptions resulted in an average detection accuracy of only 62.3%. Post-repair detection on adversarial APR patches using static analysis tools and LLM-as-judge achieved average detection accuracies of only 39.4% and 55.4%, respectively. We conclude that autonomous APR agents cannot be trusted yet in production deployment, given their susceptibility to adversarial attacks.