Medical fact verification improves with snippet level clinical grouping
MedSNIP: Building and Benchmarking Snippet-Level Granularity for Medical Fact Verification
Computation and Language
Summary
Medical claims often depend on detailed clinical context, like lab results or patient-specific factors, to be judged correctly. The authors propose checking claims by splitting them into meaningful snippet units that keep clinical details together, instead of breaking them into too small pieces. They created a new dataset and a method for automatically generating these snippets, which helps improve the accuracy of verifying medical facts. This approach reduces unnecessary checks and works especially well for complex medical claims with causal or conditional relationships.
What this means in practice
- •For clinical decision support teams: Enhance medical claim verification accuracy by preserving clinical context in snippet-level claim analysis.
- •For healthcare chatbot developers: Reduce unnecessary fact-checking calls by using automated snippet chunking for more efficient verification of long medical responses.
Authors
Hasan Iqbal, Sarfraz Ahmad, Hyunjae Kim, Sihyeon Park, Junjie Liao, Qingyu Chen, Preslav Nakov, Yuxia Wang
Abstract
A medical claim's correctness often depends not on the claim alone, but on the clinical structure around it. A claim may require a lab reference range, a causal or conditional link, or patient-specific details to be judged correctly, and atom-level decomposition can fragment these dependencies, leaving the verifier with clinically incomplete claims. We reformulate medical fact-checking around snippet-level verification, where clause-grouped units preserve local clinical structure. We introduce MedSNIP-Bench, a human-annotated benchmark for snippet-level medical fact verification, and MedSNIP, an automatic snippet-generation pipeline. MedSNIP-Bench covers 276 consumer-health and clinical-vignette responses, segmented into 2,524 snippets with dual in-general and in-patient-context labels and six structural pattern codes. MedSNIP is evaluated against human snippet boundaries on MedSNIP-Bench and then used to generate snippet-level units for external corpora. Across MedSNIP-Bench, HealthFC, and MedHallu, snippet-level verification preserves or improves false-class F1, with gains concentrated where answers are long enough to fragment and where the verifier is strong enough to exploit the recovered structure. The largest merge-pattern gain is on causal-conditional clinical chains. It also reduces verifier calls by 24-73%, though the saving survives end-to-end only when decomposition is cheap, which an open-weight decomposer makes possible at no loss of chunking fidelity.