Large language models improve detection of javascript code vulnerabilities
Exploring Automated Vulnerability Identification in JavaScript Code Using Large Language Models
Cryptography and SecurityArtificial Intelligence
Summary
Many websites use JavaScript, but its code can have security weaknesses that are hard to find. The authors studied how advanced AI language models can spot these problems better than traditional tools. They tested different AI models and ways of teaching them on many JavaScript code snippets with known security issues. They found AI models detected vulnerabilities more accurately, especially when trained further or given helpful examples. Still, these AI tools are best used along with existing methods since they don't catch all problems equally well.
What this means in practice
- •For web security teams: Improve automated scanning of JavaScript snippets for security weaknesses with AI models that outperform traditional static analysis tools.
- •For software quality assurance teams: Use AI-driven code review techniques to detect complex JavaScript vulnerabilities missed by existing security analyzers.
Authors
Manit Kaushik, Ishir Bhardwaj, Pranav Gupta, Pankaj Jalote, Arun Balaji Buduru
Abstract
JavaScript powers approximately 98.8% of all websites, making vulnerabilities in its code a significant security risk, yet existing detection approaches such as Static Application Security Testing (SAST) tools often fail to identify many real-world vulnerabilities when applied to isolated code snippets. This paper presents an empirical study of Large Language Model (LLM)-based vulnerability identification for JavaScript programs, evaluating three LLM families (Gemini 1.5 Flash, GPT-4o Mini, DeepSeek-R1-Distill-Llama-8B) across multiple prompting strategies (zero-shot, chain-of-thought, few-shot) and fine-tuning approaches on a dataset of 1,125 JavaScript code snippets spanning five Common Weakness Enumeration (CWE) categories: Injection (CWE-74), OS Command Injection (CWE-78), Cross-Site Scripting (CWE-79), SQL Injection (CWE-89), and Uncontrolled Resource Consumption (CWE-400). Our experiments show that LLMs substantially outperform traditional SAST tools on snippet-level vulnerability identification, with a fine-tuned Gemini 1.5 Flash model achieving 60% detection accuracy compared to near-zero performance from rule-based analyzers. We find that fine-tuning improves accuracy from 29% to 60%, Chain-of-Thought prompting benefits reasoning-capable models such as GPT-4o Mini, few-shot prompting is effective for polymorphic vulnerabilities such as Cross-Site Scripting, and performance varies across vulnerability categories, reaching up to 84% accuracy for structured vulnerabilities such as SQL Injection. These results indicate that LLMs provide a practical approach for automated vulnerability identification in JavaScript code, particularly when combined with task-aligned supervision, though they should complement rather than replace existing security analysis workflows due to limited recall and uneven performance across vulnerability types.