Machine learning improves detailed network intrusion detection accuracy
Multi-Class, Multi-Tier Network Intrusion Detection: A Comprehensive and Reproducible Benchmark
Cryptography and SecurityMachine Learning
Summary
Detecting cyberattacks on computer networks is tricky because existing methods often give misleading results. The authors fixed errors in a popular dataset and tested many machine learning tools to better identify types of attacks. They combined some of these tools to achieve very high accuracy in spotting attacks and telling them apart. Their setup is open for others to use and improve upon for future cyber defense work.
What this means in practice
- •For network security teams: Use the corrected benchmark and evaluation tools to improve identifying and classifying network attacks more accurately.
- •For cybersecurity product developers: Build intrusion detection systems with higher precision by incorporating the ensemble model and feature insights from the benchmark.$Commercial implications: Enables development of advanced IDS products with better attack detection and classification for enterprises.
Authors
Yufeng Xin, Bryant Goseland, Mohamed Rahouti
Abstract
Machine learning (ML) and deep learning (DL) have dominated Intrusion Detection System (IDS) research in recent years. Unfortunately, many existing studies have produced inflated results and unreliable benchmarks due to critical oversights and mistakes in the ML and DL pipeline, from data collection and labeling to feature engineering and model training and evaluation. CIC-IDS2017 is a standard benchmark for network intrusion detection. Still, many published results on this dataset are difficult to compare due to labeling errors, inconsistent flow extraction, potential leakage, and performance evaluation metrics dominated by benign traffic. In this paper, we present a comprehensive benchmark with corrected PCAP-level labeling and a complete evaluation pipeline with diverse ML models. We evaluate eleven tabular classifiers at three nested levels: binary attack detection, nine-class attack-family attribution, and fifteen-class fine-grained classification. A soft-voting ensemble of Random Forest, XGBoost, and LightGBM obtains the best fine-tier macro-F1 of 0.955, with coarse and binary macro-F1 scores of 0.980 and 0.999, respectively. We further conducted a feature selection study based on an analysis of feature importance. This comprehensive benchmark pipeline is configurable and open-source, enabling new feature extraction and model plugins for new datasets. Future work should use this pipeline as a reference point for richer features, rare-class analysis, and model generalization towards new datasets and attack classes.