Patient-centered treatment planning improves health outcomes in chronic multimorbidity
Patient-Centered Treatment Planning for Chronic Multimorbidity: A Hierarchical Reinforcement Learning Framework for Preference Modeling
Machine Learning
Summary
People with multiple chronic illnesses like hypertension and diabetes often have complex treatment plans. This paper presents a new computer method that learns treatment strategies while respecting patients' preferences on what treatments they are willing and able to follow. The method uses a hierarchical learning approach to break down decisions by disease and therapy, making it easier to understand and safer for patients. The authors show with real patient data that following this method leads to better health outcomes and respects patient choices.
What this means in practice
- •For hospital data teams: Integrate patient preference-aware reinforcement learning to support personalized treatment planning for patients with multiple chronic diseases.
- •For clinical decision support developers: Develop safer, interpretable treatment recommendation systems that prevent plans inconsistent with patient adherence capabilities and preferences.$Commercial implications: Enables building preference-respecting clinical tools that improve adherence and outcomes, which can be marketed to healthcare providers.
Authors
Nafiseh Payani, Soham Das, G. Anthony Wilson, Anahita Khojandi
Abstract
Patient preference, defined as a patient's demonstrated willingness and capacity to adhere to clinical recommendations, is a primary determinant of therapeutic effect yet remains structurally absent from existing computational treatment planning models. We address this gap by presenting patient-centered factored-action hierarchical option-critic (FAHOC), a hierarchical reinforcement learning (HRL) framework that jointly learns high-level options corresponding to therapeutic strategies and factored intra-option policies that decompose the joint action space into disease- and intervention-specific subcomponents, while imposing a cooperation-aware action masking mechanism. This enables structured exploration, improved credit assignment across hierarchy levels, and more interpretable decision pathways, while enforcing patients' preferences. Formal guarantees establish that cooperative patients achieve higher optimal expected health outcomes than non-cooperative patients, and that the factored Q-function approximation error is provably bounded. The framework is evaluated using longitudinal data collected from approximately 50,000 comorbid hypertension and type 2 diabetes mellitus patients from five hospitals in the Southeast U.S. FAHOC achieves a quality-adjusted life year expectancy equivalent improvement of 0.669 (vs -0.133 observed clinician practice), correctly identifies cooperative patients in 95.9% of cases and never violates a patient's preference in held-out test, demonstrating that HRL with explicit preference constraints can support preference-consistent, clinically safe decision-making in multimorbidity management.