Input metrics compared and improved for better software testing
An analysis of the relationship of input metrics
Software EngineeringFormal Languages and Automata Theory
Summary
Software testers use input metrics to measure how well their tests cover different input features. This paper looks at many of these metrics and finds that common ways to compare them don't work well. The authors create a better metric called k-alt-path that reduces overlap and detects differences more effectively. They then compare other input metrics systematically and suggest ways to improve future testing measures.
What this means in practice
- •For software testers: Measure and improve test suite effectiveness by using better input metrics that reduce redundancy and increase sensitivity in coverage analysis.
- •For quality assurance teams: Select appropriate input metrics to guide testing efforts systematically based on detailed comparisons of metric behaviors.
Authors
Addison Crump
Abstract
Input metrics evaluate the progress of testing in terms of features of inputs present in a test suite. Previous works, as early as the 1950s, established a number of such metrics, but few endeavored to compare them. This paper does so by utilizing existing methods proposed for other metric classes in partition testing literature. After defining and reviewing common input metrics, we begin with a short case study revealing that typical empirical comparison strategies are fundamentally insufficient for comparing metrics. Then, we demonstrate how one rigorously improves a standard metric by defining and implementing $k$-alt-path, a new metric which reduces redundancy while improving sensitivity over $k$-path. Each of the other common input metrics are then systematically compared before discussing the implications of our findings. With these contributions, we bring forward partition testing analysis methods that justify and form a strategy for future research in refining input metrics.