Streaming perception on shared edge platforms depends on workload placement
Rethinking Streaming-Perception Evaluation on Heterogeneous Edge Platforms
Distributed, Parallel, and Cluster ComputingComputer Vision and Pattern Recognition
Summary
Devices that analyze live video from multiple cameras use special processors like GPUs and NPUs to detect objects. The authors found that testing these processors one at a time doesn’t show real-world problems like delays caused by sharing the GPU with other tasks. These delays make some detections late or stale, which worsens accuracy especially for large objects. The work shows that placing workloads on the NPU rather than the GPU can sometimes improve overall detection under shared use, and that measuring average accuracy alone hides these issues.
What this means in practice
- •For edge computing engineers: Design workload placement strategies to reduce deadline misses and stale perception results on edge devices with shared GPUs and NPUs.
- •For autonomous vehicle system builders: Improve real-time multi-camera object detection by choosing processing resources that minimize latency under competing workloads.
Authors
Misun Yu, Jinyoung Moon, Jemin Lee
Abstract
Multi-camera streaming perception is increasingly deployed on heterogeneous edge platforms shared with co-resident workloads, yet accelerator placement is often evaluated using isolated single-stream experiments and mean streaming average precision (sAP). Using two end-to-end pipelines on a single GPU--NPU platform, we show that isolated evaluation can mis-rank deployment-time placement. Although the GPU pipeline is preferred in isolation, GPU-localized contention introduces deadline misses that make detections stale and can reverse the preferred placement before full GPU saturation. The NPU pipeline is less accurate than the GPU pipeline on small and medium objects in isolation, but nearly matches it on large objects. The largest absolute sAP losses in our latency and contention experiments occur for large objects. In our four-stream experiments, the preferred placement depends on which path becomes stale, and increasing GPU-side contention shifts the best placement from All-GPU to All-NPU. Under a GPU-saturating vision--language co-tenant, All-NPU achieves $5.2\times$ the worst-stream sAP of All-GPU. Because mean sAP can hide severe single-stream degradation, evaluation should report contention sweeps, deadline-miss rates on both paths, and worst-stream sAP alongside mean sAP.