Papers for

health system developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

UMLS knowledge graphs show limited gains for medical question answering

Are We Done with UMLS Injection into Language Models for Medical Question Answering?

Abstract: Maybe not. The Unified Medical Language System (UMLS) has become an oracle for millions of clinicians and health system developers worldwide. Researchers have increasingly used knowledge graphs derived from UMLS to enhance language models' factual accuracy and reasoning traceability in high-stakes tasks. Medical Question Answering (MQA) is a prime testbed for integration, with benchmarks usually involving professional exercises such as disease diagnosis and drug prescription from patient history. Despite the assumed benefits, empirical evidence shows that UMLS-based graph injection typically yields modest performance gains. This paper reviews existing MQA work, critically analyzing the selected UMLS sources and their usage with language models, and probes each of these choices in isolation. We uncover methodological issues never discussed in prior work, from low UMLS coverage to weak subgraph retrieval pipelines. Based on our findings, we provide actionable recommendations to unlock the untapped potential of UMLS as an external knowledge source.

Sat 24 OctArtificial Intelligence

The gist

Medical language models try to answer doctors' questions using an important medical database called UMLS. The authors found that adding UMLS knowledge to these models only improves their answers a little bit. They looked closely at how researchers use UMLS and found several problems like incomplete coverage and poor retrieval methods. Based on their review, they suggest better ways to use UMLS to make medical question answering more reliable.

Open → 2610.33801v1