Neuron coverage criteria help diversity but not error detection in AI models
Unraveling the Efficacy of Neuron Coverage Criteria on Testing Large Language Models
Software Engineering
Summary
Testing large AI language models to find mistakes is hard. The authors looked at methods called neuron coverage criteria, which measure how fully tests examine the model's workings. They found these criteria help check if test examples are varied and fair, but they don’t do well at spotting errors or problems the AI might make. Using these criteria to create tricky test inputs only slightly increases finding mistakes. This means current methods mostly help show test variety rather than catch AI faults.
What this means in practice
- •For ai testing engineers: Evaluate if test sets for large language models are diverse and fairly cover different types of examples using existing neuron coverage metrics.
- •For software quality teams: Decide on trade-offs in computational cost when using neuron coverage criteria for testing Transformer-based AI systems given limited error detection benefits.
Authors
Xiaoning Ren, Jiayang Song, Chongyang Liu, Jie Li, Lei Ma, Yinxing Xue
Abstract
Neuron coverage criteria aim to assess test suite quality and guide test input generation, but their effectiveness for Transformer-based language models remains unclear. We evaluate six criteria, including four established and two recent criteria, across encoder-only, decoder-only, and encoder-decoder models. Our evaluation examines error-revealing capability, distribution diversity, sample fairness (category diversity), and guidance for test input generation. The results reveal uneven effectiveness. Erroneous samples provide only 0.32% additional coverage over noise samples, indicating limited sensitivity to adversarial errors. By contrast, coverage criteria effectively assess distribution diversity and sample fairness in 26 and 30 of 33 configurations, respectively. Incorporating neuron-wise coverage guidance into test input generation increases adversarial sample production by only 4.56%. These findings suggest that existing coverage criteria can serve as an antidote for assessing diversity but remain largely a placebo for revealing adversarial errors, particularly when used individually. Their practical value therefore depends on whether these limited benefits justify the computational overhead of testing large Transformer-based models. Our study highlights the need for more robust and efficient coverage criteria tailored to these models, with stronger error-revealing capability and greater value for test generation.