Event sparse transformer cuts hardware cost for malware detection
ESL-former: Event Sparse Linear Transformer for FPGA-Accelerated Malware Detection
Hardware Architecture
Summary
Detecting malware on small, power-limited devices is challenging because the usual methods need too much computing power. The authors present a new type of transformer model that identifies important parts of data without doing expensive calculations, making it easier for simple chips to process. They also designed a hardware system that efficiently handles this trimmed data so the device runs faster and uses less energy. Tests show their method keeps good detection accuracy while using fewer resources.
What this means in practice
- •For embedded security engineers: Build malware detection systems on low power FPGAs that maintain accuracy while reducing hardware cost and latency.
- •For fpga hardware designers: Design efficient hardware accelerators that handle sparse transformer data streams to optimize memory and computation on edge devices.
Authors
Jun Yang, Yumeng Cao, Yingxuan Xiao, Changhong Li, Shangru Zhou
Abstract
Transformer based intrusion detection has shown strong potential for identifying unknown malware variants, but its deployment on resource constrained edge devices remains difficult due to the high computational and memory cost of self attention. This paper presents an event sparse Linear Transformer framework for malware detection on FPGA edge platforms. Instead of relying on an explicit attention matrix, the proposed method derives token and channel importance directly from the accumulated key value state of Linear Transformer. A contribution aware scoring function is introduced to measure the perturbation of each token or kernel channel to the linear attention state, allowing redundant tokens and low contribution channels to be removed without constructing quadratic attention maps. To translate sparsity into practical hardware acceleration, the proposed design further replaces conventional binary mask based execution with a Top K active token buffer and a hardware compaction unit. The selected active tokens and channels are reorganized into dense fixed length data streams, enabling downstream processing elements to execute regular multiply accumulate operations with reduced redundant memory access and control overhead. This design is suitable for low power FPGA based malware detection, where both latency and on chip resource usage are critical. Experimental evaluation demonstrates that the proposed sparse Linear Transformer can reduce hardware cost while preserving malware classification accuracy.