LLM evaluation method improves software engineering test reliability
SafeLLM4SE: Statistical Evaluation and Reporting for LLM-based Software Engineering Systems
Software EngineeringMachine Learning
Summary
Software tools powered by large language models (LLMs) can produce different outputs each time they run, making it hard to judge how reliable they are. The authors introduce SafeLLM4SE, a way to measure and report these tools' results statistically, showing not just average performance but also variability and uncertainty. This helps developers understand and compare LLM-based software testing more clearly. They also provide a free software package so others can use this method easily.
What this means in practice
- •For software testing teams: Use statistical methods to evaluate LLM-generated code tests for clearer assessment of tool reliability and variability in outputs.
- •For ai development teams: Apply SafeLLM4SE to compare different LLMs fairly by accounting for randomness in their software engineering tasks.
Authors
Francisco Ortin
Abstract
Large language models (LLMs) are increasingly used for software engineering tasks, yet their stochastic behavior challenges the validity, reproducibility, and comparability of their evaluations. Conventional practices such as reporting a single output, an average score, best-of-N, or pass@k performance can obscure variability and estimation uncertainty, potentially leading to misleading conclusions about system reliability. This article presents SafeLLM4SE, a practical methodology and reporting standard for statistically principled evaluation of LLM-based software engineering systems. Rather than treating generated outputs as deterministic artifacts, SafeLLM4SE treats them as realizations of a stochastic process and distinguishes quality, stability, and estimation uncertainty. It combines adaptive sampling with confidence intervals, distribution-aware statistical comparisons, effect sizes, and a minimum reporting standard covering model configuration, reproducibility, evaluation procedures, and resource usage. SafeLLM4SE is also provided as an open-source software package available on PyPI, enabling researchers and practitioners to reproduce and extend the methodology. We illustrate its application by comparing two LLMs on HumanEval, a benchmark of programming problems assessed through functional tests.