Gpu takes control to speed up network ai without cpu delays
Toward Intelligent Networks via AI-Aware GPU-Native Packet Processing
Networking and Internet Architecture
Summary
Network systems often slow down because CPUs have to handle every packet before AI can analyze it, causing delays. The authors present AGP, a new method that lets GPUs directly process network packets and run AI models without waiting for the CPU. This setup greatly reduces delays and uses less power while keeping up with high-speed data. The authors showed that their system can detect threats much faster with lower resource use compared to traditional methods.
What this means in practice
- •For network operations teams: Implement gpu-only packet processing to reduce latency and boost throughput in network intrusion detection systems.
- •For data center infrastructure managers: Deploy gpu-native frameworks to enable energy-efficient high-speed AI-based network monitoring without cpu involvement.
Authors
Seyed Mohammad Mehdi Mirnajafizadeh, Yiwen Hu, Rhongho Jang
Abstract
The integration of inline Artificial Intelligence (AI) models into critical network infrastructure is fundamentally bottlenecked by the high latency and synchronization overhead of CPU-mediated packet processing. While legacy GPU offload and recent CPU-bypass frameworks attempt to bridge this gap, they remain trapped in proprietary ecosystems or still rely on the host CPU and coarse-grained batching to coordinate stateful telemetry and AI pipeline execution. In this paper, we propose AGP, a novel framework that promotes the GPU from a passive accelerator to a primary data-path controller, answering what changes architecturally when the GPU autonomously owns the complete packet-to-inference pipeline. By enabling GPU-native packet processing, contention-free in-GPU stateful aggregation, and a persistent mega-kernel for continuous AI inference, AGP removes the CPU from the critical path. Our evaluation demonstrates that native in-GPU packet processing sustains line-rate throughput while being over 6.1x more power-efficient. Furthermore, our integrated Intrusion Detection System (IDS) stress test eliminates the legacy CPU-GPU synchronization tax, reducing end-to-end latency by 7.9x (up to 35x p99) and accelerating whole-system inference throughput by 9.3x using only ~2% of the GPU's thread capacity.