SkillDRE improves malicious agent skills using staged defense feedback
SkillDRE: Dual-Stage Red-Team Evolution of Agent Skills via Pre-Execution and Runtime Feedback
Cryptography and SecurityArtificial Intelligence
Summary
Some computer programs called agents use skill packages to perform tasks. These skills can be attacked and improved by hackers who try to sneak harmful code past security checks. The authors present SkillDRE, a system that cleverly changes these skills using two levels of feedback: one before the code runs and one during its execution. This helps attackers make their harmful skills work better without being detected, while still doing the original good task. Their tests show SkillDRE is more effective than previous methods at bypassing defenses.
What this means in practice
- •For security testers: Enhance automated attack simulations by evolving attack strategies using feedback from both code scanning and runtime defenses.
- •For cyber defense teams: Improve defense evaluation by exposing vulnerabilities that pass initial code checks but fail under runtime monitoring.
Authors
Pengyu Zhu, Jingyi Yang, Yi Liu, Li Sun, Sen Su
Abstract
Agent skills package instructions, executable code, and task-specific resources into reusable artifacts that agents can improve using execution feedback. The same mechanism also enables attackers to evolve malicious skills, making them more effective and less detectable. However, a candidate skill may pass pre-execution scanning yet fail to realize its target under runtime defenses, while a revision that repairs execution may introduce new scanner findings. We introduce SkillDRE, a fully automated framework for evolving complete malicious skill packages through a dual-stage feedback loop. Given a benign task and its associated skills, SkillDRE autonomously constructs and validates a task-conditioned malicious objective and a verifiable judge rule. It then holds both fixed while evolving the skill implementation, with preservation of legitimate task capability. SkillDRE combines scanner-guided evolution with runtime-guided refinement informed by execution outcomes observed under runtime defense. Each runtime-guided revision returns to the pre-execution stage for rescanning and further optimization before re-execution, forming a cross-stage closed loop. Evaluated on SkillsBench across four victim models, SkillDRE achieves an average attack success rate of 45.28%, exceeding the strongest baseline by 40.3%, while its final submitted skills receive no SkillScan findings and largely preserve benign-task performance. These results show that two-stage defense feedback can serve as a useful learning signal for adaptive red teaming and that evaluating either defense stage in isolation can miss the resulting attack capability. Codes is available at https://github.com/whfeLingYu/SkillDRE