FASE: Fast Adaptive Semantic Entropy for Code Quality
2026-06-08 • Software Engineering
Software EngineeringArtificial IntelligenceMultiagent Systems
AI summaryⓘ
The authors focus on improving how multiple AI agents collaborate to write software by better measuring when the generated code is likely correct or uncertain. They propose a new metric called FASE, which estimates code correctness using a smart method involving graphs, instead of costly checks done by large language models (LLMs). Their tests show that FASE predicts correctness more accurately than previous methods, while being much faster and cheaper to run. This makes FASE a useful tool for making multi-agent coding systems more reliable and efficient.
multi-agent systemcode generationlarge language modelsemantic entropyuncertainty quantificationminimum spanning treegraph theoryfunctional correctnessSpearman correlationcomputational overhead
Authors
Shizhe Lin, Ladan Tahvildari
Abstract
Multi-agent code generation offers a promising paradigm for autonomous software development by simulating the human software engineering lifecycle. However, system reliability remains hindered by LLM hallucinations and error propagation across interacting agents. While semantic entropy provides a principled way to quantify uncertainty without ground-truth answers, current methods often rely on costly LLM-driven equivalence checks. In this work, we introduce Fast Adaptive Semantic Entropy (FASE), a novel metric that approximates functional correctness based on the minimum spanning tree of structural and semantic dissimilarity graphs. Evaluations on HumanEval and BigCodeBench demonstrate that FASE outperforms state-of-the-art semantic entropy by LLM entailment, achieving a 25% average improvement in Spearman correlation and a 19% increase in ROCAUC score against Pass@1 from ground-truth test cases when using the Qwen3-Embedding-8B model. Furthermore, by eliminating costly LLM-driven equivalence evaluation, FASE incurs negligible computational overhead, requiring only approximately 0.3% of the runtime cost of traditional semantic entropy approaches. These results position FASE as a practical, cost-effective solution for optimizing uncertainty quantification in real-world multi-agent workflows.