Calibrated embedding distances reveal reliability limits in ai vulnerability detection
Diagnosing Generalization Boundaries in AI-Based Vulnerability Detection via Calibrated Embedding Distances
Software EngineeringMachine Learning
Summary
AI tools for finding security problems in code often seem accurate but can fail when the code changes in certain ways. The authors show that measuring how code examples relate to each other in the AI's hidden feature space helps spot when the AI might make mistakes. They tested this idea on different types of changes, like switching projects or time periods, finding that some changes cause bigger drops in accuracy. This method helps diagnose when and why the AI’s predictions become unreliable without needing extra models.
What this means in practice
- •For software security teams: Assess the trustworthiness of AI vulnerability alerts by analyzing embedding distances to spot unreliable predictions under code changes.
- •For code analysis tool developers: Incorporate calibrated embedding distances into AI models to detect when vulnerability predictions may fail across different projects or time periods.
Authors
Konstantin Ievtushenko, Ralf Lämmel, Jan Jürjens
Abstract
High benchmark scores in learning-based vulnerability detection mask severe degradation under distribution shifts. This paper demonstrates that calibrated embedding geometry exposes these hidden generalization boundaries on syntactically similar vulnerable and patched code pairs. An evaluation of CodeBERT and Qwen2.5-Coder across pattern, project, and temporal shifts on a strictly paired dataset reveals three main findings. First, whitened intermediate-layer embeddings with local nearest-neighbour scoring separate unfamiliar samples with a mean out-of-distribution AUROC of 0.739 (RQ1). Second, distance scores reveal a shift-specific reliability gradient where accuracy drops by over 35 percentage points and cross-project shifts cause the sharpest degradation, even though temporal shifts drift geometrically farthest (RQ2). Third, false positives on patched code systematically correlate with geometric proximity to vulnerable training neighbourhoods, increasing from 9.4% in the lowest-risk decile to 58.8% in the highest (RQ3). These calibrated distances establish a diagnostic signal that maps representation geometry directly to model degradation without requiring auxiliary models.