Budget-aware language model improves medical risk predictions with verifiable knowledge

Cite What You Explore: Budget-Aware LLM Reasoning over Medical KGs with Verifiable Evidence

Machine Learning

Summary

Predicting health risks after patients leave the hospital is hard because medical records don’t show all important links between diseases and treatments. The authors created a system called BAR that uses large language models to explore medical knowledge graphs carefully and within a budget. BAR retrieves good-quality evidence to support its predictions and clearly shows where its information comes from, making it easier for doctors to check the reasoning. This approach improves accuracy and trustworthiness while keeping the cost of retrieving information low.

What this means in practice

  • For hospital data teams: Improve post-discharge risk predictions by integrating verifiable external medical knowledge with controlled computational costs.
  • For clinical decision support developers: Build safer and more transparent AI systems for healthcare by enabling models to cite and verify medical evidence used in predictions.

Authors

Chen Chen, Dongjie Wang, Mei Liu, Zijun Yao

Abstract

Post-discharge risk prediction from electronic health records (EHRs) is difficult because many dependencies that link discharge-time observations to downstream complications, such as comorbidity cascades and drug-disease interactions, are absent from the record. External medical knowledge graphs (KGs) can supply these missing dependencies, but tracing them demands three properties: KG exploration must remain cost-bounded, retrieved evidence must be differentiated by source quality, and the resulting rationale must be citable for retrospective review. Large language models (LLMs) can plan and verify over structured evidence, making them natural candidates for KG reasoning, but existing LLM-based methods do not satisfy these three properties jointly. In this paper, we propose BAR, a Budget-Aware LLM Reasoning framework over medical KGs with three contributions. First, BAR refines the raw KG into disease-specific evidence graphs whose edges carry support scores and provenance records, turning the KG into a quality-annotated reasoning space rather than a static feature source. Second, an LLM then reasons over this graph through a plan-navigate-verify loop that decomposes the question into steps, retrieves evidence under a patient-specific budget, and revises when verification fails. Third, a reasoning policy is trained with a reward that compares predictions with and without acquired evidence, combined with acquisition cost and citation-integrity terms. Across 8 diseases and 3 prediction horizons on MIMIC-III and MIMIC-IV, BAR improves AUPRC by 3.4 points over the strongest baseline, raises citation precision from 59.8% to 77.9%, and consumes only 62-65% of the budget cap.