Style based malware detection improves anomaly ranking and analysis
SAGEGAN: Style-Based Anomaly Detection with Gaussian Embeddings using Generative Adversarial Networks
Cryptography and Security
Summary
Malware changes quickly, making it hard for traditional rules to catch all threats. The authors developed SAGEGAN, a system that learns what normal software looks like using only safe, non-malicious files. It turns software files into special images to understand their patterns and spots unusual files that might be malware. This approach helps not only detect malware but also understand how malware differs from normal software. Their method works well on different sets of malware without needing retraining.
What this means in practice
- •For security operations teams: Detect new and evolved malware by identifying unusual software patterns without relying on known signatures.
- •For cyber threat analysts: Analyze and compare malware families by inspecting how their software characteristics deviate from normal, aiding investigation and response.
Authors
Thesath Wijayasiri, Kar Wai Fok, Vrizlynn L. L. Thing
Abstract
Malware evolves faster than rule-based and signature-driven detection pipelines. This paper presents SAGEGAN, a benign-only trained malware anomaly detection framework that converts portable executable files into compact three-channel images and models benign structure through style-conditioned adversarial reconstruction. The representation combines Hilbert-mapped byte values, benign-referenced byte-transition surprise, and entropy deviation from benign software. The model encodes each image into a layer-wise style tensor aligned with a seven-stage modulated generator, rather than a single latent bottleneck. A Gaussian style prior, moment-based prior alignment, and latent consistency are used to reduce mismatch between encoded benign styles and the generator's sampled manifold. For interpretation, a deterministic encoder pathway maps each executable to a fixed style tensor, enabling repeatable layer-wise family distance, gradient sensitivity, principal component, and class-behaviour analyses. On a self-collected portable executable corpus containing malware from 214 families, the Gaussian variant achieves 89.76% area under the receiver operating characteristic curve and 88.19% balanced accuracy, while the genome-style variant reaches 88.03% and 84.09%, respectively. Without refitting model weights, benign reference statistics, or decision thresholds, the same checkpoints are evaluated on DIKE, Microsoft BIG 2015, and Lester malware subsets. The results suggest that layer-wise style modelling supports both anomaly ranking and structured post hoc analysis of how malware families depart from the benign manifold.