Delta Debugging in the Absence of Test Oracles Through Metamorphic Testing
2026-07-01 • Software Engineering
Software Engineering
AI summaryⓘ
The authors address a problem in delta debugging, which is a method to simplify program inputs while keeping certain behaviors, but it usually needs a way to check if the simplified input still works correctly. Since some programs lack clear ways to verify correctness (no test oracle), the authors propose DDMT, a new approach that uses metamorphic testing to check properties without needing a traditional oracle. They tested DDMT on 66 programs and found it often works better or as well as existing methods, especially in situations where no test oracle is available. This makes delta debugging more useful in tricky cases where correctness isn't easy to define.
Delta debuggingTest oracleOracle problemOracle-deficient programsMetamorphic testingInput reductionProperty preservationTest functionQuery efficiencyProgram debugging
Authors
Mingyue Jiang, Yongqiang Tian, Tsong Yueh Chen
Abstract
Delta debugging provides an automatic way to minimize a program input while preserving a certain property. However, its effectiveness fundamentally relies on the availability of test oracles to determine whether a reduced input still preserves the specific property. Consequently, the oracle problem substantially limits the applicability of existing delta debugging techniques, particularly for oracle-deficient programs where output correctness cannot be directly determined. To address this problem, this paper proposes a novel approach, DDMT, to enhance the applicability of delta debugging, especially facilitating its application to oracle-deficient programs. Our key insight is to redesign an oracle-independent test function and incorporate it into the reduction procedure of delta debugging such that the property-preservation validation can be accomplished without requiring a test oracle. To this end, DDMT employs the technique of metamorphic testing, which is a property-based and oracle-independent testing method. It establishes a metamorphic testing-based test function, using it as a replacement for the original test function adopted by delta debugging. The experiments evaluate DDMT on 66 subjects across both oracle-available and oracle-deficient scenarios, with different delta debugging approaches. The results positively confirm that DDMT can enhance the applicability of delta debugging while often preserving or improving reduction effectiveness and query efficiency. Furthermore, compared to the relevant delta debugging approaches, DDMT is also able to achieve performance improvements with proper configurations.