Error supervising neural network detects CNN parameter faults
ESupNNet: An Error Supervising Neural Network architecture for error detection against soft errors in parameters
Hardware Architecture
Summary
Deep learning models like CNNs can sometimes make mistakes because of tiny errors in their settings caused by things like hardware glitches. The authors created a small extra neural network that watches the main CNN’s behavior to catch when these errors might cause a wrong answer. This helper network uses relationships between categories the CNN knows to spot problems without changing the original model. They tested this system on several popular image recognition models and datasets and found it detects errors with over 90% accuracy.
What this means in practice
- •For deep learning engineers: Add lightweight error detection modules to deployed CNN models to catch parameter corruption without modifying original architectures.
- •For hardware reliability teams: Use an error supervising network to monitor soft errors in AI accelerators running CNNs, improving system trustworthiness.
Authors
Jorge Cano-Paez, Luis Entrena, Almudena Lindoso
Abstract
This work presents a novel approach to detect misclassification errors in CNNs caused by soft errors in their parameters. We propose an architecture that uses inter-class relations induced by the CNN that needs protection. The architecture has minimal resources overhead and does not require modifying the CNN, which makes it a competent solution that can be used with other error protection techniques. We have validated the architecture with five different combinations of modern dataset-model pairs: ImageNet-1K for ResNet-50 and EfficientNetV2-Small; CIFAR-10 for MobileNetV3, ShuffleNetV2-Small with 2.0x output channels and MNASNet with depth multiplier of 1.3. The validation process was done rigorously with statistical significance, from the creation of the datasets used by the architecture to the acquisition of experimental results. Results show great performance with over 90% accuracy in detecting single errors and great error detection over multiple Bit Error Rates, which can be potentially increased with hyperparameter tuning.