Evaluating Open-Weight LLMs for Generating Structured Threat Information for Autonomous Vehicle Vulnerabilities
2026-07-17 • Cryptography and Security
Cryptography and SecurityArtificial Intelligence
AI summaryⓘ
The authors looked at how self-driving cars have many parts that can be hacked, and these problems are usually written in plain text, which is hard to use for fixing issues. They tested several large language models (LLMs) to automatically turn these plain-text vulnerability reports into a structured format called STIX, which helps organize threat information better. Their models did well identifying some parts of the threats but still struggled with others, like attack techniques. They also found common threat patterns in self-driving cars, showing how AI can help improve security by making the information easier to understand and act on.
Connected and Autonomous Vehicles (CAV)Common Vulnerabilities and Exposures (CVE)Structured Threat Information Expression (STIX)Large Language Models (LLMs)Common Weakness Enumeration (CWE)MITRE ATT&CKThreat IntelligenceElectronic Control UnitsVulnerability Mapping
Authors
Md Erfan, Ahmed Ryan, Md Kamal Hossain Chowdhury, Md Rayhanur Rahman
Abstract
Connected and Autonomous Vehicles (CAVs) rely on interconnected software and hardware components, including sensors, Electronic Control Units, in-vehicle infotainment systems, and telematics units, where vulnerabilities can compromise assets, users, and vehicle operations. These vulnerabilities are commonly documented as plain text in the Common Vulnerabilities and Exposures (CVE) database; however, security practitioners require structured information about affected assets, types of weaknesses, and attack behaviors to effectively mitigate the risks from these vulnerabilities. To this end, we evaluate open-weight Large Language Models (LLMs) for generating Structured Threat Information Expression (STIX), a well-known structured format for representing threat information, for CAV-related CVEs. We construct a dataset called CAV-STIXGen that maps CAV vulnerability descriptions to STIX domain objects (SDO), STIX relationship objects (SRO), Common Weakness Enumeration (CWE), and MITRE ATT&CK techniques mappings. Using this dataset, we evaluated 11 open-weight LLMs (4B to 120B parameters) across various prompting strategies and temperatures. Single-model configurations achieve F1 scores of 0.94 for SDO, 0.63 for SRO, and 0.99 for CWE mapping, while complete MITRE ATT&CK mapping remains challenging. In a multi-agent setup, Gemma-4-31B and Codestral-22B achieve F1 scores of 0.91 for SDOs and 0.43 for SROs, respectively. Lastly, we analyze CWE and MITRE ATT&CK co-occurrences to identify recurring threat patterns in the CAV domain, demonstrating how AI-assisted vulnerability-to-STIX translation can automate threat intelligence and prioritize defense in transportation security.