CoG-Guided Weight Correction for Fault-Tolerant Deep Neural Networks
Machine LearningHardware Architecture
Summary
The authors address the problem that deep neural networks (DNNs) used in important tasks can have their performance damaged by hardware faults affecting their internal numbers (weights). They propose a new method that fixes these faulty weights by looking at their positions within each network layer, without needing to retrain the model or change its design. Their experiments on models used in medical applications and popular image recognition networks show that their method greatly improves fault tolerance while keeping accuracy almost the same. This is the first time the concept called Center of Gravity has been used to improve neural network reliability in this way.
Authors
Bahram Parchekani, Samira Nazari, Ali Azarpeyvand, Mohammad Hasan Ahmadilivani, Tara Ghasempouri, Jaan Raik
Abstract
Deep Neural Networks (DNNs) used in safety-critical applications are vulnerable to hardware and memory faults that corrupt network weights and degrade reliability. In this paper, we propose a Center of Gravity (CoG) guided weight correction method that restores faulty weights based on their spatial characteristics within each layer. The proposed approach detects and corrects weight faults using distance-aware correction rules, eliminating the need for retraining or architectural modification. The effectiveness of the proposed method in terms of the capability of tolerating hardware faults has been evaluated through performing fault injection at different Bit Error Rates (BERs). Experiments on safety-critical LSTM-based Networks, including StageNet for disease progression tracking and MTFNet for cardiac anomaly detection, demonstrate fault tolerance improvements of up to 230x and 6.41x, respectively, at a BER of 10^{-3}, with negligible accuracy loss. When extended to Convolutional Neural Networks (CNNs), the method achieves up to 49.55x and 20.79x improvements under comparable fault conditions on ResNet-18 and VGG-16, respectively. To the best of our knowledge, this is the first work to apply the CoG concept to neural network weight tensors for enhancing model reliability.