Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detection
2026-06-29 • Cryptography and Security
Cryptography and SecurityArtificial Intelligence
AI summaryⓘ
The authors studied how big AI language models check computer code for security problems and found these models can be tricked by mental shortcuts, similar to how humans can be biased. They created tests that kept the code the same but changed the surrounding information to see if it influenced the models' decisions. Results showed the models often made different vulnerability judgments based on this extra context, especially when the problem needed deeper understanding. The authors also showed it’s possible to exploit these biases to hide security issues in code from the models.
Large Language ModelsVulnerability DetectionCognitive HeuristicsHalo EffectFraming EffectAnchoring EffectSemantic ReasoningCode SecurityBlack-box Attack
Authors
Asif Shahriar, Hongyu Cai, Hadjer Benkraouda, Gang Wang, Z. Berkay Celik
Abstract
Researchers and practitioners increasingly apply Large Language Models (LLMs) for automated vulnerability detection. Recent work has shown that LLMs are susceptible to the same cognitive heuristics that bias human judgment. Yet, no work has investigated whether these heuristics affect a model's assessment of code vulnerabilities. In this paper, we present the first systematic exploration of cognitive heuristics in LLM-driven code vulnerability detection. We introduce a controlled framework that holds the code fixed and only varies the surrounding context to trigger three cognitive heuristics: the halo effect through author attribution, the framing effect through task objectives and consequences, and the anchoring effect through prior analysis results. Within this framework, we evaluate eight LLMs across three programming languages and perform both quantitative and code-level analyses. Our findings demonstrate that all evaluated models are susceptible to these heuristics. Cross-model average susceptibility is highest for framing at 33.2%, followed by anchoring at 23.5% and halo at 18.4%. Code-level analysis reveals that vulnerabilities that require semantic reasoning for detection are more susceptible to cognitive heuristics than those identifiable through pattern matching. Furthermore, models often change their verdict from safe to vulnerable based on the cognitive condition, without accurately identifying the actual vulnerability. To highlight the practical impact, we demonstrate a proof-of-concept black-box cognitive attack that can suppress up to 97% of previously detected vulnerabilities. These findings indicate that cognitive susceptibility is a consistent and exploitable property of LLM-based vulnerability detection.