Auditable dataset reveals compiler security bug patterns in c code
CISB-Bench: An Auditable Source--IR Dataset of Compiler-Introduced Security Bugs
Cryptography and SecuritySoftware Engineering
Summary
Compiler-introduced security bugs happen when the tools that turn code into programs accidentally change how safe the program is. These bugs are hard to study because the proof is spread out in many places. The authors collected 429 exact examples from two big compilers, with all the details needed to check and learn from them. They tagged each example carefully to show which ones really are security bugs and which are not. This dataset helps others understand and find these compiler-caused security bugs more easily.
What this means in practice
- •For compiler developers: Use the dataset to identify and analyze real security bugs introduced during compilation to improve compiler reliability and safety features.
- •For software security teams: Employ the dataset artifacts to detect and prevent security risks caused by compiler behavior changes in production software builds.
Authors
Saatvik Pradhan, Anuroop Saini, Pranav Attrey, Suman Saha
Abstract
Compiler-introduced security bugs (CISBs) arise when an optimization, lowering, or instrumentation decision changes a security-relevant property of the generated program. They are difficult to study because their evidence is distributed across issue reports, reduced tests, historical configurations, and compiler artifacts; a security-related report also does not imply that every associated reduction establishes a security-bearing compiler failure. We present CISB-Bench, an auditable dataset of 429 exact C-program rows mined from GCC and LLVM. Each row contains its C reduction, a standardized LLVM IR analysis bundle at -O0 through -O3, public provenance, a final binary label, and a primary mechanism or boundary annotation. Two reviewers independently labeled the fixed corpus, agreeing on 369 rows (86.0%, Cohen's kappa=0.662); the 60 disagreements were adjudicated. The final dataset comprises 280 CISBs and 149 hard non-CISB cases. The prediction task is to recover this reviewed exact-row label from the supplied artifacts; it is not a claim that standardized IR alone reproduces every historical compiler failure. We characterize the security mechanisms and evidence boundaries represented by the corpus, and demonstrate how its paired artifacts support source-only, IR-aware, and joint analyses. CISB-Bench provides a reusable, inspectable target for compiler-security mining and detection research.