Propagating construction-time knowledge quality into medical question answering: A framework grounded in clinical guidelines
Artificial Intelligence
Summary
The authors studied how to improve the use of information extracted from medical guidelines to answer questions better. They created a method that scores how good each piece of information is, based on its structure and evidence, and keeps this quality score to help select and present the best answers later. Testing on Chinese diabetes guidelines, they showed that using these quality scores reduces missing knowledge and contradictory answers while improving accuracy. Clinicians preferred answers using their full method over simpler ones, showing the approach helps make medical question answering more reliable.
Authors
Jie Hu, Junjie Wang, Shan Lu, Yifang Hu, Gong Cheng, Yun Liu
Abstract
Large language models have facilitated knowledge graph (KG) construction from clinical guidelines, but extracted triples vary in structural validity and evidential support. Meanwhile, graph-augmented question answering (QA) systems typically optimize query relevance during retrieval, with limited reuse of quality information produced during KG construction. This creates a disconnect between construction-time quality control and inference-time evidence use. We investigate whether construction-time triple quality can serve as a persistent signal for downstream evidence selection and presentation. We propose a quality-aware framework that models structural conformance (SchemaConf) and evidential support (EvidScore) as complementary dimensions and fuses them into a per-triple quality signal, Q(t). Rather than using quality solely for filtering, the framework retains Q(t) and derived quality tiers as graph attributes and propagates them into quality-weighted subgraph retrieval and tier-conditioned evidence prompting, while preserving passage-level provenance. Experiments on Chinese diabetes clinical guidelines show that the utility of the quality signal is distribution dependent. Under cross-version and cross-model shift, the fused Q(t) provides stronger triple-quality discrimination than either component alone (AUC 0.748 vs. 0.703 for EvidScore and 0.645 for SchemaConf). In guideline-grounded QA, propagating construction-time quality reduces required-knowledge omission from 16.3% to 5.3% and conflicting outputs from 16.3% to 2.7%, with an evidence-grounded precision of 81.6% and near-zero invalid citations. Blinded clinician ratings favor the full framework over no retrieval (4.68 vs. 4.21 on a five-point scale) and approach the oracle condition (4.80), while cross-generator experiments show consistent trends.