Vulnsage connects ai reasoning with code to spot real security bugs
VulnSAGE: Adjudicating SAST Reports by Connecting LLM Reasoning to Program Structure
Software EngineeringCryptography and SecurityProgramming Languages
Summary
Many computer security tools raise alerts about potential problems, but often these are false alarms that waste developers’ time. The authors created VulnSAGE, a system that uses artificial intelligence combined with program analysis to better decide which alerts are actual security defects. It carefully examines the code context and checks the AI’s reasoning for accuracy. Tested on thousands of alerts from popular code projects, VulnSAGE was able to identify real security bugs more accurately and helped developers fix previously unknown issues.
What this means in practice
- •For software security teams: Identify and confirm real security vulnerabilities among many static analysis alerts to reduce false alarms and guide effective fixes.
- •For open source maintainers: Better prioritize and verify vulnerability reports on large public C/C++ projects to improve security maintenance.
Authors
Jinbao Chen, Zixiang Zhou, Yu Zhang, Yihong Wu
Abstract
Static Application Security Testing (SAST) reports flag operations whose safety depends on project-specific context, and maintainers must decide which reports describe real defects. Large language models (LLMs) can interpret this context, but adjudicating a report with them requires the decisive code, which often lies far from the reported line, and a decision basis that program analysis can check. We present VulnSAGE, an adjudication system for SAST reports that connects LLM reasoning to program structure in both directions. Its report-centered slicer recovers the dependencies of the reported operation together with the program structure that explains them, such as the caller object on which a helper acts. CLAIM, an intermediate representation for report judgments, records each model interpretation as a semantic fact that cites the program relations it assumes. Program analysis checks these premises, and the verdict follows from the facts whose premises hold. Both mechanisms share a program graph, which lets the model cite the code it reads and lets the checker query those citations. Statically supported true positives then guide proof-of-concept generation and runtime reproduction. Across 187 labeled reports from four corpora covering five vulnerability families, VulnSAGE rejects 89 of 97 false alarms and retains 85 of 90 real defects. Under a common PoC procedure, 25 of its positive verdicts reproduce at runtime, compared with three for the LLM-based adjudicator LLM4SA. Applied to 45,440 alerts raised by five SAST tools on the 200 most-starred C/C++ projects on GitHub, it reproduces 104 defects at runtime. Maintainers have confirmed 24 of the 26 defects they have responded to, including ten previously unknown defects, nine of which are already fixed.