LLM framework extracts lung cancer tumor stages with transparency
SIFTING: A Novel LLM-Based Framework for Structured and Transparent Information Extraction from Clinical Free-Text Reports, with Application to Tumor Staging in Lung Cancer
Computation and Language
Summary
Extracting detailed medical information from free-text clinical reports is hard because such data is often unstructured and difficult to verify. The authors created a system called SIFTING that uses advanced language models to carefully pull out specific tumor stage details from lung cancer reports, while linking each fact back to where it appears in the text. This makes the output more accurate and easier to check. Their system performed as well as experienced clinicians at this task, using a self-hosted language model that preserves data control.
What this means in practice
- •For hospital data teams: Implement structured extraction of lung cancer tumor stages from radiology reports to enable easier validation and use in clinical workflows.
- •For medical software developers: Build clinical information extraction tools that provide transparent, verifiable output by linking extracted data segments back to source texts.
Authors
Mirco Hess, Gerben van Veenendaal, Joris Wakkie, Yiwen Soo, Malcolm H. Lawson, John D. Maclay, Arjun Nair, Neal Navani
Abstract
Background: Large language models (LLMs) show promise for extracting information from clinical free-text documents, but their outputs are often unstructured and lack traceability, complicating validation and adoption in clinical workflows. In this work we introduce SIFTING, an LLM-based framework designed to address these shortcomings. Methods: SIFTING combines the language comprehension capabilities of LLMs with segment-level processing and structured prompts with strict output control, linking findings to the source text to enable both accurate and transparent information extraction. To demonstrate its capabilities, we applied the framework to the task of extracting tumor T-stage information from 130 lung cancer radiology reports (SIFTING-T-stage). A compact 4-bit quantized version of the open-source LLM Llama-3.3-70B (35 GB) was used in a fully self-hosted setup, providing full control over data and model. Performance was evaluated against a reference standard created by four clinical experts and compared with a range of LLMs as used in a conventional single-prompt approach, using bootstrap resampling to estimate confidence intervals. Results: SIFTING-T-stage achieved an accuracy of 90% (95% CI: 84-95) against the reference standard. We found its performance to be comparable to even the largest state-of-the-art LLMs with reasoning capabilities and to be interchangeable with clinical experts (p < 0.001), while at the same time offering full traceability through source text references. Conclusion: SIFTING enables accurate, structured, and traceable information extraction from clinical free-text documents. It ensures data control, reproducibility, and verifiable outputs that can support clinical validation and workflow integration.