Lightweight error detection improves CNN safety on embedded GPUs
Carry-Through Checksum: A Lightweight Fault-Detection for CNN Inference at the Edge
Hardware ArchitectureMachine Learning
Summary
CNNs help computers understand images but sometimes make mistakes due to tiny hardware errors, especially on small devices. The authors propose a new method called carry-through checksum that uses the CNN’s own calculations to check for errors without slowing it down much. It can catch most serious mistakes and fix them quickly by re-running the calculation. This keeps the CNN working reliably even on low-power GPUs in safety-critical situations.
What this means in practice
- •For edge ai developers: Detect and fix silent errors in CNN inference on embedded GPUs with minimal delay and resource use.
- •For autonomous vehicle engineers: Ensure reliable CNN outputs for safety-critical edge devices running vision tasks with low overhead error detection.
Authors
Kyrylo Nazarevych, Mohammad Hasan Ahmadilivani, Krister Kaldre, Davide Bertozzi, Jaan Raik
Abstract
Convolutional Neural Networks (CNNs) are increasingly deployed in safety-critical edge applications, where soft errors can silently corrupt inference outputs and lead to unsafe decisions. Such applications typically rely on resource-constrained embedded GPUs, requiring fault detection and mitigation techniques that add minimal compute, memory, and latency overhead while integrating seamlessly with the standard GPU inference pipeline. Existing algorithm-based fault tolerance techniques rely on matrix augmentation and per-operation checksum verification, imposing substantial overhead that is prohibitive for CNN inference on embedded GPUs. In this work, we propose carry-through checksum, a fundamentally new scheme for soft-error detection in CNN inference on embedded GPUs. The method embeds dedicated carry-through filters into the convolutional layers, which compute a checksum from the CNN's own operations and propagate it through inference, enabling end-to-end error detection with a single output verification. Experimental results on multiple CNN architectures show that the proposed method detects 95.86% and 86.56% of critical faults for FP32 and FP16, respectively, at almost no additional per-image overhead. Detected faults are mitigated through re-execution, incurring only 2.27% run-time overhead across the entire test set on an NVIDIA Jetson Orin NX GPU.