Care sd improves local language models accuracy and reliability in security interviews
CARE-SD: Explainable Reasoning Architecture for Local SLM Self-Disclosure Analysis in Textual Data
Cryptography and SecurityArtificial IntelligenceComputation and Language
Summary
It can be hard to tell when someone is genuinely sharing personal security information in interviews because meanings depend on context and overlap in categories. The authors created CARE-SD, a system that helps smaller language models better understand and explain these personal disclosures without sending sensitive data outside local devices. CARE-SD made smaller models nearly as accurate as much larger ones while reducing mistakes in claims made about what was said. This could help protect privacy while analyzing security conversations.
What this means in practice
- •For cybersecurity analysts: Automatically and reliably identify personal security disclosures in sensitive interviews without sharing data externally.
- •For privacy software developers: Build local data analysis tools that protect participant privacy while improving interpretation accuracy using enhanced small language models.$Commercial implications: Enables privacy-compliant software products that analyze sensitive text data without cloud dependency, ensuring confidentiality for organizations.
Authors
Aisvarya Adeseye, Adeyemi Adeseye, Mohammad Tahir, Jouni Isoaho
Abstract
Cybersecurity interviews uncover critical security experiences, behaviors, and privacy concerns often not fully captured by conventional technical assessments. Consequently, identifying genuine self-disclosures is challenging because statements are context-dependent, categories overlap, and automated interpretations may introduce unsupported claims. Locally deployed Small Language Models (SLMs) protect sensitive participant data by keeping processing within controlled environments, but their limited reasoning capabilities make simple prompting insufficient for context-aware and factually reliable analysis. This study developed CARE-SD (Context-Adaptive Reasoning and Explanation for Self-Disclosure), a novel four-module architecture integrating context-adaptive processing, hierarchical disclosure reasoning, explanation-consistency verification, and cross-batch reconciliation. We evaluated three local SLMs, including Llama 1B, Qwen 1.5B, and Llama 3B, using 82 independently annotated cybersecurity interviews across five organizational sectors. Comparisons included conventional classifiers, three prompting strategies, larger 8B--32B models, and architectural ablations. Two independent reviewers assessed disclosure labels, factual claims, and explanation correctness. CARE-SD improved the accuracy and evidence reliability of SLMs, increasing Llama 3B Macro-F1 from 0.80 to 0.88 while reducing unsupported claims by 60\%. Remarkably, the 3B model matched the 20B reference model in category Macro-F1, demonstrating the effectiveness of the architecture without requiring a larger model.