Frequency domain mixing boosts malware detection in network traffic

Frequency-Domain Mixing Data Augmentation for Malicious Traffic Detection

Cryptography and Security

Summary

Malicious software often hides in network data that changes a lot, making it hard for computer programs to spot it. The authors found that usual training data isn't diverse enough, so these programs struggle with new or different types of threats. They improved a technique that mixes parts of data in the frequency domain, which means they alter the data in a way that keeps important details but adds variety. This helps detection models learn better and catch bad traffic more reliably across different network conditions. Tests showed their method works better than other ways to make training data more diverse.

network trafficmalicious traffic detectiondata augmentationfrequency domainmachine learningout-of-distribution generalizationdeep learningtraining data diversityfrequency-domain mixing

Authors

Yuhao Yan, Bo Lang, Xiangyu Li

Abstract

The strong dynamics of network traffic often force malicious traffic detection models to handle out-of-distribution data. Typically, deep learning-based malicious traffic detection models require a large amount of high-quality training data. However, owing to challenges such as high labeling difficulty and resource consumption, existing datasets often suffer from insufficient diversity and fail to capture evolving traffic patterns, leading to poor out-of-distribution generalization ability of the trained models. Data augmentation has been widely adopted to improve data diversity and model generalization. Recently, frequency-domain mixing augmentation has shown promising performance because it effectively perturbs data while preserving key structural information. This approach shows potential for enhancing malicious traffic detection models. However, existing studies lack theoretical interpretation of the mixing mechanism, and do not adapt to the characteristics of network traffic. In this paper, we first conduct a theoretical analysis of the current frequency-domain mixing method, revealing its underlying principles and limitations. We further propose an improved frequency-domain mixing-based data augmentation method for network traffic data, which enhances the diversity of sequence features in network traffic and improves the out-of-distribution generalization of malicious traffic detection models. Extensive experiments on multiple artificial and real-world datasets demonstrate that our method substantially improves detection performance across diverse network environments and outperforms other data augmentation approaches.