Radiation-hardened neural networks that self-repair after bit flips
A Radiation-Hardened, Fault-Aware Quantized Neural Network That Self-Heals
Machine Learning
Summary
Neural networks used in places with lots of radiation, like space, can get their data messed up by high-energy particles. The authors developed a way for these networks to fix themselves when parts of their stored data get corrupted, without needing extra copies of the data. Their method uses a smart technique to identify and repair damaged parts by focusing on the healthy data components. This approach improves the networks’ reliability and extends the time they can operate correctly compared with traditional error correction methods.
What this means in practice
- •For spacecraft engineers: Extend operational lifetime of neural networks running on radiation-exposed spacecraft hardware by repairing corrupted weights at runtime without extra memory overhead.
- •For data center operators: Improve resilience of neural networks in upcoming space-based data centers by integrating self-healing techniques that correct bit errors dynamically.
Authors
Peter Xiangyuan Ma, Joshua S. Bloom
Abstract
Deep learning (DL) models are increasingly being deployed into radiation-challenging environments in spacecraft, particle accelerators, and, soon, space-based data centers. Instantiated as a learned orchestration of numbers stored in persistent memory, models are susceptible to continual degradation as high-energy particles corrupt data. At inference time, new data passed through the network fundamentally break the off-line assumptions and guarantees of a static model determined during training. To preserve the quality and efficacy of the models, fault tolerance against bit flips is a clear objective. In this work we developed a runtime method for repairing the quantized weights of neural networks (NNs) after bit flips on commodity, commercial off-the-shelf (COTS) hardware, without storing extra replicas. Our work extends the well-established Fault-Aware Training (FAT) paradigm by adding an alignment penalty to concentrate undamaged activations and weights into a low-rank subspace, and we then reconstruct flagged corrupted coordinates from the uncorrupted survivors. We call this method {\it Projection Healing} and apply it to six regression benchmarks on quantized NNs. Holding a FAT checkpoint fixed and toggling only the runtime heal gives a further ${\sim}23\%$ reduction in the native-unit excess integral. Across those datasets the method extends the operational scrub interval by a median of $2.04\times$ at a $+5\%$ RMSE degradation and $1.96\times$ at $+50\%$, compared with $1.62\times$ and $1.07\times$ for classical Single Error Correction, Double Error Detection (SECDED). This introduces a new path towards designing resilient data-driven models, complementing established hardware redundancy, ECC, and FAT-based techniques.