Validating LLM-Modernized Scientific Software Through Differential Fault Injection
2026-08-14 • Distributed, Parallel, and Cluster Computing
Distributed, Parallel, and Cluster ComputingSoftware Engineering
AI summaryⓘ
The authors studied whether modernizing old scientific software code with large language models (LLMs) still keeps the software's behavior reliable, especially when errors or changes occur. They tested this by applying controlled faults to both the original and updated versions of a key code part in the GAMESS software and compared the results. Their tests showed that the modernized code behaved just like the original when facing faults, and also revealed some issues related to parallel processing and precision. Overall, the authors demonstrate a method to check if code modernization preserves fault responses.
large language modelscode modernizationfault injectionGAMESSself-consistent-fieldscientific computingtransient faultspersistent perturbationsparallel deadlocksreduced precision
Authors
Evan Coleman, Yuzhong Shen, Masha Sosonkina, Peng Xu
Abstract
Large language model (LLM) agents are increasingly used to modernize the legacy Fortran underlying production scientific software, but validation of these transformations emphasizes nominal executions and may not test whether a modernization preserves the original code's response to faults, perturbations, and reduced precision. We present a differential fault-injection validation method: a harness instruments the shared self-consistent-field driver of GAMESS at twelve sites and applies identical, deterministic faults to the original and LLM-modernized implementations, isolating the converted integral kernels. Across more than 2,200 runs, transient-fault absorption costs match a contraction-based model (predicted slopes 0.74 and 1.49 iterations per bit; measured 0.82 and 1.50), persistent perturbations halve final-energy error per additional bit, and the campaigns expose phase-dependent parallel deadlocks and false convergence under reduced precision. The original and modernized kernels agree in all 200 paired injections, and a measurement-guided synchronization change composes with the modernization, matching in all 40 pairs.