Fusion of data views has limited impact on encrypted command detection
Multi-View Fusion for Encrypted C2 Detection: A Leakage-Controlled Measurement Study of Evaluation Pitfalls
Cryptography and SecurityArtificial Intelligence
Summary
Detecting hidden command-and-control messages in encrypted internet traffic is tricky. This paper studied how combining two types of traffic data affects detection accuracy and found that common evaluation mistakes can hugely overstate performance. The authors show that many flows aren't independent, and known malicious addresses are missing in some samples, making detection seem easier than it is. Even combining data views helps only a little, and attackers can fool all models tested. The key takeaway is that careful evaluation design matters most for this detection task.
What this means in practice
- •For network security teams: Use the study’s evaluation insights to avoid inflated detection scores when monitoring encrypted C2 traffic in enterprise networks.
- •For cyber threat detection developers: Improve encrypted C2 detection tools by incorporating correct cross-validation and accounting for data grouping highlighted in this paper.
Authors
Hoang-Huy Nguyen-Huu, Van-Tri Phan, Khuong Nguyen-An
Abstract
Command-and-control (C2) traffic increasingly hides within TLS, so defenders now apply machine learning to traffic metadata. Many studies assume that combining two metadata views, namely flow statistics and TLS handshake fingerprints, improves both accuracy and robustness. We tested this assumption on 17,577 TLS flows from 62 real Cobalt Strike captures. Our evaluation removes the data leakage that leads to overly optimistic reported scores. We report three findings that matter more than the fusion result itself. First, an incorrect preprocessing step increases the F1 score by 0.28. This step computes the frequency encoding across the entire dataset rather than within each cross-validation fold. The increase is about ten times larger than any real effect we measured. Second, both the labels and the behavioral features depend on the destination address. Because of this, the 17,577 flows form only 2,132 independent groups, and the positive rate of 55.1\%, which looks balanced, drops to 4.2\%. Therefore, class balance is just a result of how we analyze the data, specifically whether we count flows or endpoints, and not a real feature of the task. Third, 20 of the 62 captures (32\%) have no TLS flows to any known C2 address, so they contain only benign samples. We checked these captures directly and confirmed that this is a gap in the ground truth, not a labeling error. In this context, fusion beats the best single view by only 0.022 in F1. When an attacker forges both feature surfaces simultaneously, every model performs worse than a simple baseline that always predicts positive (F1 = 0.711). For encrypted C2 detection, the evaluation design is not a preliminary step. It \emph{is} the main result.