Quantum compilers hide nearly a third of circuit layout bugs
What Output-Equivalence Oracles Miss: An Empirical Study of Equivalence-Invisible Bug Fixes in Quantum Transpilers (Qiskit, tket, Cirq)
Software Engineering
Summary
Quantum compilers translate quantum programs into hardware instructions and are usually checked for correctness by comparing their outputs. The authors found that many bug fixes in popular quantum compilers fix problems that these standard checks miss, especially issues related to how circuits are mapped to qubits or recorded. This means current testing methods can overlook certain subtle bugs. They confirmed this problem across different quantum compiler tools and shared their data for further study.
What this means in practice
- •For quantum software developers: Detect and fix bugs in quantum compilers that standard output tests miss, improving compiler reliability and correctness.
- •For quantum hardware integrators: Evaluate the fidelity of quantum circuit mappings to hardware layouts to catch subtle compiler errors overlooked by usual verification methods.
Authors
Furqan Nasir, Arif Shah, Iftikhar Alam
Abstract
Quantum compilers are judged correct by an output-equivalence oracle: the compiled circuit must compute the same unitary as the original, modulo global phase and a qubit-layout permutation. This oracle, by construction, checks only that semantic map, not the circuit's own layout, permutation, or phase records: a defect there, or in a fixed-seed run's determinism, can pass unseen though the record is public. We measure how often this happens in real merged compiler fixes: a systematically identified corpus of Qiskit transpiler bug-fixes, classified by an independently dual-coded, source-validated manifestation taxonomy. Nineteen of 68 fixes (28%, 95% Wilson CI 19-40%) repair faults that this equivalence screen does not catch, even one augmented with compilation-validity, circuit-quality, and performance checks. A secondary wider-window robustness sample produced a nearly identical fraction (29/104, 27.9%, CI 20-37%). A conservative floor remains even restricted to the one unconditionally equivalence-invisible channel (a corrupted layout or permutation record): 10 of 68 fixes (15%, CI 8-25%) beneath the 28% headline. The gap is not Qiskit-specific: it replicates in tket (7 of 21, 33%), with Cirq smaller but consistent. We detected no systematic differences on five inexpensive PR-level characteristics (19 vs 49, underpowered to rule out a modest true difference). This class dominates the invisible set, concentrating at representation-boundary crossings. We release the corpus, codebook, and coding artifacts. Here we only measure it.