Large language models show growing tolerance for hardware errors
Fault-tolerant foundation models
Machine LearningArtificial IntelligenceHardware Architecture
Summary
Computers often make small mistakes when they try to save energy, which can cause problems when running complex software like large language models (LLMs). The authors show that these LLMs can actually learn to handle such errors better as they get bigger, rather than getting worse. They trained many models on simulated faulty hardware and found that models seem to use special error-correcting strategies inside themselves. This discovery suggests we might run AI on less reliable but more energy-efficient hardware without losing performance.
What this means in practice
- •For data center operators: Run AI inference workloads on lower-power, less reliable hardware by relying on models trained to tolerate errors, reducing energy costs.
- •For mobile device engineers: Deploy large language models capable of coping with hardware faults to maintain functionality on energy-constrained mobile or edge devices.
Tested on simulated data.
Authors
Trevor McCourt, Ila R. Fiete, Isaac L. Chuang
Abstract
Emerging computer hardware often trades reliability for energy efficiency; here we show that large-language models (LLMs) can be trained to tolerate this unreliability, and that rather than degrading, their error resilience actually increases as they grow. Modified neural scaling laws inferred from 40,000 GPU-hours of training runs on simulated faulty digital hardware quantify this trend and suggest that models learn to compute within "good" error-correcting codes, whose relative overhead remains finite no matter how large the model gets. This finding leads us to conjecture that appropriately trained LLMs may be formally fault-tolerant; if true, running AI inference on low energy, faulty hardware may be a path to substantial energy savings over the status quo.