cybersecurity tool developers

Abstract: Operational Technology (OT) environments require realistic, reproducible security datasets, yet existing approaches often lack automation, multi-source data capture, and sufficient documentation for reuse. This paper presents ISIA-AF, a modular attack framework for orchestrating reproducible attack execution and automated dataset generation on industrial systems. The framework coordinates distributed attack clients, records network traffic and operational data, ultimately leading to a multi-source dataset. We derive functional and non-functional requirements from prior work and stakeholder discussions, and realise the framework following a design science research approach. A case study on the ISIA testbed, comprising a real industrial system and a simulated process, demonstrates how the framework supports centralised control, low communication overhead, and flexible deployment across network segments. The result is a practical basis for generating extensible, multi-source OT security datasets for intrusion detection research.

Abstract: Users increasingly turn to Large Language Models to answer a variety of questions, including cybersecurity questions. We study how personalization strategies can help improve the effectiveness of answers to questions asked to an LLM-based cybersecurity assistant. Beyond accuracy, we focus on the understandability, actionability and, most importantly, motivating power of answers, given how often users fail to follow cybersecurity recommendations. Specifically, we investigate four personalization strategies, ranging from static user profiles to interaction-history-based personalization, using a corpus of 1,045 real-world cybersecurity questions and a 7-day deployment involving 57 participants and 1,066 user questions. Across both a large-scale automated LLM-based evaluation and human evaluation, conversation-based personalization is consistently favored in comparative ratings of perceived helpfulness and likelihood of following security advice. Importantly, the relative trends observed in the LLM-based evaluation align with those obtained from human evaluation, suggesting that LLM-based evaluation can provide a scalable mechanism for comparing personalization strategies before costly user studies. These results indicate that behavior-driven personalization is a promising direction for LLM-powered cybersecurity assistants and highlight the value of combining LLM-based and human evaluation when studying personalized language-model systems.