Small language models show promise for generating malware code tasks
From Billion-Scale to Million-Scale Language Models: Operational Capabilities for Embedded Post-Exploitation
Cryptography and SecurityArtificial Intelligence
Summary
This paper looks at whether very small language models, which use less computer power, can help create software tools for hackers after they break into a system. The researchers tested seven small language models to see how well they could write code for two tasks related to hacking. They found that bigger models don’t always do better and that how the task is described to the model matters a lot more than just how the questions are asked. Although these small models still need too much computing to be used by malware widely today, improvements could make that easier in the future.
What this means in practice
- •For cybersecurity defense teams: Develop and anticipate new AI-assisted hacking techniques by understanding the capabilities of small language models in generating offensive code.
- •For embedded systems developers: Design countermeasures for resource-constrained devices vulnerable to autonomous AI-driven malware using insights about small language model capabilities.
Authors
Gabriel Hortea, Simon Pietro Romano, Juan Tapiador, Mario Varlese
Abstract
Small Language Models (SLMs) are increasingly attractive for resource-constrained environments due to their low computational footprint. These characteristics also make them candidates for future autonomous malware, where connectivity, latency, privacy, or operational constraints may preclude cloud-based inference. Yet the ability of ultra-compact models to support malware during post-exploitation remains poorly understood. We systematically evaluate seven open-source SLMs (20M-1.5B parameters) on two post-exploitation code-generation tasks: Directory Enumeration and Backdoor Generation. We compare Python (high-resource) and Lua (low-resource) across prompting strategies, task formulations, iterative reasoning, grammar-constrained decoding, and execution-feedback loops. Results show that offensive code-generation capabilities do not increase linearly with model size. Successful execution depends on the interaction of model capacity, task formulation, and prompting strategy, with task formulation exerting a greater influence than prompt engineering alone. Algorithmic formulation combined with few-shot prompting enables Qwen2.5-Coder 1.5B to reach Pass@5 of 100% in both Python and Lua for Directory Enumeration, 100% in Python for Backdoor Generation, and 99.2% in Lua. Although the evaluated models remain too computationally demanding for widespread deployment in contemporary malware, rapid progress in compact models may reduce this barrier. These findings help anticipate AI-assisted malware and inform future countermeasures.