Code analysis method detects AI errors before running code
CodeTD: Topology of Attention Detects Hallucinations in Code LLMs
Software EngineeringArtificial IntelligenceComputation and LanguageMachine Learning
Summary
AI tools that write computer code sometimes make mistakes called hallucinations, producing code that doesn’t work or has serious security problems. The authors created CodeTD, a way to check if generated code is likely correct without running it, by studying patterns in how the AI pays attention to input details. They use a math technique called topological data analysis on these attention patterns to spot mismatches. Tests on popular coding benchmarks and multiple programming languages showed CodeTD works better than other methods and can be used across different coding tasks.
code LLMshallucinationstopological data analysisattention mapscode correctnesspre-execution assessmentprompt-generation mismatchbenchmarksAI-code assistantsprogramming languages
Authors
Daria Voronkova, Ilya Trofimov, Anton Dmitriev, Eduard Tulchinskii, Evgeny Burnaev, Serguei Barannikov
Abstract
As AI-code assistant tools become widespread, automatic assessment of the correctness of generated code becomes a significant challenge. Code LLMs are prone to hallucinations, which may lead to code that does not solve the required problem, or even to code with severe security vulnerabilities. In this paper, we introduce CodeTD -- the first approach to pre-execution assessment of code correctness based on topological data analysis (TDA) of Code LLMs' attention maps. Our method quantifies prompt-generation mismatch using topological patterns of attention maps. We carry out experiments with common benchmarks (HumanEval, MBPP, BigCodeBench, MultiPL-E), 5 programming languages and 10 Code LLMs of size up to 34B parameters. The experimental results show that the proposed method outperforms recent baselines. Moreover, CodeTD is transferable between coding benchmarks.