Improving edge AI reliability with smart DNN partitioning under faults
RAPID-DNN: Reliability-Aware Partitioning for Robust DNN Inference Deployment on Edge AI
Emerging Technologies
Summary
Edge AI devices run deep neural networks but often face hardware faults that can scramble data and reduce accuracy. The authors created RAPID DNN, a system that carefully splits DNN tasks to balance speed, energy use, and fault tolerance. It figures out which parts of the network are most sensitive to errors and optimizes deployment accordingly, improving accuracy even when hardware errors happen. Tests on popular neural networks and edge AI hardware models show notable accuracy improvements with only small extra delay and energy costs.
What this means in practice
- •For edge ai device engineers: Deploy DNN inference models more reliably on resource-limited edge accelerators by optimizing inference partitioning for fault tolerance alongside performance.
- •For hardware accelerator designers: Design hardware-aware DNN deployment strategies that consider runtime fault behaviors to improve system-level robustness.
Authors
Mukta Debnath, Krishnendu Guha, Debasri Saha, Amlan Chakrabarti, Susmita Sur-Kolay
Abstract
Deep neural networks (DNNs) deployed on heterogeneous edge accelerators are increasingly exposed to runtime hardware faults. Tight power and resource budgets push these platforms toward aggressive voltage scaling and limit the use of hardware fault mitigation, while process variation, thermal stress, and device aging raise fault rates further. The resulting transient bit level corruptions propagate through inference and can sharply reduce prediction accuracy. Existing DNN partitioning frameworks optimize latency and energy under fault free assumptions, so their deployment strategies often fail to sustain reliable inference in realistic operating conditions. This paper presents RAPID DNN, a reliability aware partitioning framework that adds runtime reliability as a third optimization objective alongside latency and energy. RAPID DNN first characterizes layer wise reliability through systematic fault injection in the weight and activation domains to identify vulnerability critical layers. The resulting sensitivity profile guides a three objective NSGA II optimizer that jointly minimizes inference latency, energy consumption, and expected fault induced accuracy degradation. We evaluate RAPID DNN on AlexNet, SqueezeNet, ResNet18, VGG16, and MobileNetV2 using heterogeneous Eyeriss and SIMBA accelerator profiles. Across fault models and injection probabilities, it consistently improves inference robustness over the fault agnostic baseline. Under the representative mixed random fault configuration, RAPID DNN raises average Top 1 accuracy by 9.81 percentage points and reduces average expected fault induced accuracy degradation by 36.86%, at a cost of 2.38% average latency overhead and 3.34% average energy overhead. These results show that treating reliability as a partitioning objective enables robust DNN deployment on heterogeneous edge accelerators under runtime hardware faults.