Oss supply chain threats detected with ai guided by attack types
Autonomous OSS Threat Detection via Taxonomy-Aligned LLMs
Cryptography and Security
Summary
Open source software faces many tricky attacks that can sneak in harmful code or confuse developers. The authors created a new way to detect these attacks automatically by using a big language model trained to recognize specific types of threats. They made a special list of attack kinds and gathered nearly a thousand real examples from past incidents. Their approach can tell which kind of attack it is with very high accuracy, better than other machine learning methods. The key is that the AI is guided by the organized list of threats rather than just learning blindly.
What this means in practice
- •For software security teams: Detect and classify different OSS supply chain threats automatically using AI aligned with an attack taxonomy to improve response times and accuracy.
- •For devops engineers: Integrate taxonomy-aligned large language model prompts to spot malicious activities in build pipelines and dependency management processes.
Authors
Md. Robiul Islam Niloy
Abstract
Open source software (OSS) ecosystems face growing threats from sophisticated supply chain attacks including typosquatting, dependency confusion, Trojan Source obfuscation, malicious build injection, and CI/CD pipeline poisoning. Existing detection approaches rely on signature-based tools and rule-based systems that struggle to generalize across attack variants and emerging threat patterns. In this paper we propose a taxonomy-aligned large language model framework for automated detection and classification of OSS supply chain threats. We introduce a structured AV-xxx threat taxonomy covering five attack categories and construct a curated dataset of 999 verified real-world OSS supply chain incidents sourced from GitHub Security Advisories, CISA alerts, and security research reports spanning 2018 to 2026. Using taxonomy-aligned prompt engineering with GPT-4, our framework achieves 97.0\% multi-class classification accuracy and 97.0\% macro F1 score across all five threat categories. Comparative evaluation against five traditional machine learning baselines, one zero-shot open source LLM, and two fine-tuned neural models reveals a surprising finding: fine-tuned Llama 3.1 8B (70.5%) and SecRoBERTa (77.5%) both underperform simple TF-IDF classifiers (82.3%), while Mistral 7B without taxonomy alignment achieves only 65.7%. These results confirm that taxonomy-aligned prompting rather than model scale, domain pretraining, or fine-tuning is the critical factor enabling high classification accuracy. Our dataset and code are publicly available to support reproducible supply chain security research.