Summary
Medical billing is complex because it involves strict rules and free-text explanations. The authors created a system that first uses fixed rules to check codes and limits, then uses AI to understand the supporting documents. Their method makes clear where decisions come from and forces the AI to only support claims when exact evidence is found. Tests on synthetic data showed mixed results compared to simpler approaches, but requiring explicit evidence helped catch missing information. More testing on real data and with humans is needed to see if this approach is practical.
What this means in practice
- •For healthcare billing teams: Verify medical billing claims by combining strict rule checks with AI-supported document analysis while keeping audit trails clear.
- •For compliance auditors: Inspect exactly which rules and document evidence support or contradict each claimed medical billing item for transparent audits.
Tested on simulated data.
Authors
Jan Hölter, Kevin Geis, Benjamin Raab, Boris Bauke
Abstract
This work presents a proof of concept for auditable LLM-assisted medical billing verification based on a decision-basis contract. The contract separates deterministic checks of versioned fee-catalog rules from LLM-based assessment of free-text documentation. The deterministic layer resolves the applicable catalog release and checks code availability, quantity limits, and exclusions. The semantic layer classifies each claimed item as supported, contradicted, or missing required information. Support and contradiction require a verbatim evidence span; unavailable rule context, unsuccessful assessment, or missing required evidence prevents support through fail-closed abstention. We evaluated four locally run open-weight models on a synthetic catalog and 36 curated cases under the contract, an ablation without explicit documentation requirements, and an end-to-end baseline. Outcome agreement varied across models and showed no consistent advantage over the baseline. Explicit documentation requirements improved identification of missing information for all four models. The evidence gate also exposed cases in which correct raw judgments lacked valid evidence and were converted to incomplete decision-basis entries. The results show how explicit decision records can make rule findings, documentation judgments, and abstention reasons inspectable. Evaluation on real catalogs, independently annotated documentation, and with human reviewers is required to assess practical value.