Can Large Language Models Recover Semantic Optimization Opportunities That Compilers Miss?
2026-08-04 • Programming Languages
Programming LanguagesArtificial Intelligence
AI summaryⓘ
The authors study whether large language models (LLMs) can help compilers find useful program improvements that they usually miss because some important details are not explicitly in the code. They created a special test set called SeGaBench, with various cases that include hidden program meanings and tools to check if improvements are correct and faster. Testing several LLMs, they found the best model often produced correct improvements and speed-ups but didn’t always match the best possible optimizations. This suggests LLMs can help compilers by suggesting possible improvements, as long as these suggestions are carefully checked.
compiler optimizationlarge language modelsprogram semanticsperformance benchmarkingC/C++semantic validationdata-structure invariantsexecutable benchmarkprogram transformationoracle artifact
Authors
Hailong Jiang, Feng Yu, Emran Hossain, Jianfeng Zhu, Mengfei Ren, Qiang Guan, Chunwei Xia
Abstract
Optimizing compilers miss profitable transformations when their enabling semantics are absent from the analyzed program representation. We ask whether large language models (LLMs) can recover such semantics from heterogeneous C/C++ context and realize them as validated, contract-preserving artifacts. We introduce SeGaBench, an executable benchmark containing 100 synthetic and 20 source-backed cases spanning low-level assumptions, data-structure invariants, and high-level semantic lifting. Each case includes hidden enabling semantics, an oracle artifact, correctness and semantic validators, and a reproducible performance protocol. We evaluate five LLMs using five independent responses per case. The strongest model produces correct artifacts in 94.8% of responses, achieves at least 1.05x speedup in 83.3%, and obtains a performance success on 93.3% of cases. Nevertheless, correct artifacts often close only part of the oracle gap. These results show that LLMs can complement compiler analysis as speculative semantic proposers, provided that their artifacts are validated and evaluated.