MedFailBench: A Clinician-Built Open-Source Benchmark for Medical AI Safety Boundary Inspection
Artificial IntelligenceComputation and Language
Summary
The authors created MedFailBench, a new tool to check not just if medical AI gives the right answer, but specifically what kind of safety mistake it makes. They worked with doctors to build a set of 44 example cases, each labeled by how serious the error is and what safety rule it broke. The tool helps researchers understand AI failures better by organizing errors into categories like missing urgent care or giving unsafe advice. Their current release includes detailed guidelines, test cases, and a system to evaluate AI responses without using real patient data.
Authors
Goktug Ozkan
Abstract
Most medical AI benchmarks measure whether a model knows the correct answer. MedFailBench asks a different question: which safety boundary failed? We present a clinician-built synthetic benchmark and failure atlas that labels medical AI errors by severity (1--5) and safety gate type (missed urgent escalation, unsafe remote dosing, unsafe discharge reassurance, evidence fabrication, unsafe protocol execution, source support gap). The current public release (v0.2.1) contains 44 clinician-reviewed synthetic cases with severity annotations, a live HuggingFace leaderboard preview, a safety gate taxonomy, a clinical severity rubric, and an automated pipeline for archiving model-response screening runs. No patient data, clinical validation claims, or model rankings are included. MedFailBench is released under Apache-2.0 and CC-BY-4.0 and carries the Zenodo DOI 10.5281/zenodo.21205535.