Bastet: A Fine-Grained Expert-Labeled Dataset for DeFi Smart Contract Vulnerability Detection
2026-06-02 • Cryptography and Security
Cryptography and Security
AI summaryⓘ
The authors address problems in existing datasets used to find vulnerabilities in DeFi smart contracts, such as outdated code versions, unreliable automated labels, and overly simple classifications. They created Bastet, a new dataset based on real audit reports from 2021 to 2024, with expert human annotations to improve accuracy. Bastet uses a detailed two-layer labeling system to better capture complex vulnerability types. This work aims to provide more reliable data to help detect and prevent security issues in modern DeFi contracts.
DeFismart contractvulnerability detectionSoliditylabel noisedataset annotationCode4renawhite-hat securitytaxonomyaudit reports
Authors
Wan-Hsuan Hsu, Wei-Hsin Wang, Cheng-Yu Liou, Ting-Rui Ke, Kentaroh Toyoda
Abstract
Smart contract vulnerabilities in Decentralized Finance (DeFi) protocols resulted in over 1.49 billion USD in confirmed losses in 2024 alone, across 192 incidents [1]. As LLM-based vulnerability detection emerges as a promising approach to address these threats, the quality of evaluation datasets has become a critical bottleneck. Existing datasets suffer from three fundamental problems: they are built on outdated Solidity versions (e.g., v0.4) that no longer reflect modern DeFi contracts [5][6][7]; they rely on automated or LLM-generated annotations that introduce hallucination-driven label noise [9][10]; and they apply coarse single-layer labeling that fails to capture the semantic complexity of real-world business logic vulnerabilities [6][7][11][12]. We present Bastet, an expert-labeled DeFi smart contract vulnerability dataset that addresses all three problems through real-world audit findings (2021-2024), human expert annotation with discussion-based consensus, and a two-layer taxonomy of 46 Tags and 77 Subtags. Bastet comprises 4,402 findings collected from 394 Code4rena competitive audit reports spanning April 2021 to November 2024, of which 849 findings are fully annotated by white-hat security researchers from the DeFiHackLabs community. All annotations are produced through a two-annotator consensus workflow, ensuring label accuracy grounded in real-world vulnerability root causes.