Coding agents tricked to use attacker packages via malicious rules
Package Hallucination Attacks on Coding Agents through Prompt Injection in Rule Files
Cryptography and SecurityArtificial Intelligence
Summary
Coding agents use shared rule files to help write code automatically, but these files can be secretly changed by attackers to trick the agents. The attackers inject hidden instructions that cause agents to swap safe code packages for bad ones controlled by the attacker. The authors created a system called PackHallu that learns how to write these sneaky instructions effectively. Their tests show many coding agents can be fooled this way, exposing a new security risk.
What this means in practice
- •For software security teams: Detect and block malicious prompts injected in rule files that lead coding agents to use harmful external packages.
- •For devops engineers: Improve integrity checks on rule files guiding automated code generation to prevent dependency manipulation attacks.
Authors
Yupu Wang, Zhengyuan Jiang, Reachal Wang, Neil Zhenqiang Gong
Abstract
Modern agentic coding frameworks increasingly rely on community-shared rule files (e.g., AGENTS.md or .cursorrules) to guide autonomous code generation, yet the security risks of this pipeline remain underexplored. To bridge this gap, we introduce the package hallucination attack, where an attacker injects malicious prompts into benign rule files to induce coding agents to replace legitimate dependencies with attacker-controlled packages. To obtain effective malicious prompts injected into rule files, we propose PackHallu, an evolutionary optimization framework that iteratively rewrites these injected prompts using trajectory-level feedback and LLM-guided mutations. Evaluations across multiple benchmarks, LLMs, and agent frameworks show that PackHallu achieves high attack success rates and strong transferability across diverse models and agent combinations. Our findings demonstrate that coding agents are vulnerable to package hallucination attacks, highlighting the urgent need for stronger security safeguards in autonomous coding systems.