LLMs for Zero-Shot Threat Detection via Structured Risk Indicators
2026-08-17 • Cryptography and Security
Cryptography and SecurityMachine LearningNetworking and Internet Architecture
AI summaryⓘ
The authors propose a two-step method using large language models (LLMs) to detect insider threats and advanced cyber attacks without needing prior examples. Their approach turns user activity logs into easy-to-understand risk indicators before analyzing attack patterns over time. They tested this on two datasets and found their method works better than previous models, especially when adding relevant past data helps weaker models. The study shows that creating high-quality risk indicators is crucial for spotting threats accurately in this zero-shot setting.
Large Language Model (LLM)Zero-shot detectionInsider threatsAdvanced Persistent Threats (APTs)Retrieval-Augmented Generation (RAG)Security logsRisk indicatorsChronological timelinesF1-scoreCyber threat detection
Authors
Abdullah Alghamdi, Siamak Layeghy, Marius Portmann
Abstract
We propose a two-stage large language model (LLM) framework for zero-shot detection of insider threats and advanced persistent threats (APTs) from heterogeneous security logs. The framework models user activity as chronological timelines and incorporates retrieval-augmented generation (RAG) to provide personalised behavioural context from each user's historical activity. Rather than performing end-to-end classification directly from raw logs, it first generates structured, interpretable sets of threat-specific risk indicators, which are then classified jointly across temporal sequences to capture attack patterns spanning multiple windows.The framework is evaluated on two benchmark datasets, CERT r5.2 for insider threat detection and PicoDomain for APT detection, using four combinations of two open-weight LLMs under both retrieval and non-retrieval settings. All configurations outperform the previous state-of-the-art LLM-based framework (GABM), with the best configuration improving the F1-score by 11.40 percentage points on CERT r5.2 and 31.50 percentage points on PicoDomain. Results further show that retrieval mainly benefits weaker LLMs by generating more discriminative risk indicators, whereas stronger models achieve comparable performance without retrieved context. The most effective assignment of LLMs to the two stages depends on the dataset. These findings show that the quality of the generated risk indicators is the main driver of zero-shot cyber threat detection performance.