MedGym:A Unified Continuous-Time Benchmark for Dynamic Medical Treatment Reinforcement Learning
2026-05-31 • Machine Learning
Machine Learning
AI summaryⓘ
The authors created MedGym, a tool to better test how well computer programs can recommend medical treatments that change over time and vary by patient. Unlike previous tools that only look at fixed time points, MedGym models patient health continuously and handles irregular treatment schedules. This helps researchers compare different methods more realistically, including how safe and personalized the treatments are. MedGym also lets studies run both with past data and in real time, providing a thorough way to evaluate treatment recommendations.
reinforcement learningcontinuous-time modelingdynamic treatment recommendationMarkov decision processPhysics-Informed Neural Networksoffline learningonline learningpersonalized medicinetrajectory safetybenchmark environment
Authors
Yuepeng Wang, Ken Kawano, Yongqi Zhou, Yoshihiko Fujisawa, Richard Weiss, Akifumi Wachi, Katsuki Fujisawa, Ying Chen, Mehrshad Sadria, Xin Liu, Kyoung-Sook Kim, Xiao Hu, Sebastien Gros, Xun Shen
Abstract
Medical treatment recommendation poses several challenges to reinforcement learning (RL): patient physiology evolves in continuous time, measurements and interventions are performed at irregular intervals, and treatment effects vary substantially across individuals. Existing RL formulations and simulated environments, however, are based on discrete-time MDP or POMDP abstractions with fixed or pre-specified decision intervals. Thus, it remains difficult to evaluate whether RL methods can handle time-interval-dependent disease progression, personalized treatment response, and safety between consecutive measurement points. To address this gap, we introduce MedGym, a benchmark environment for dynamic treatment recommendation. MedGym models longitudinal patient evolution in a continuous-time framework and constructs a configurable medical RL benchmark from clinical data by using Physics-Informed Neural Networks. The resulting benchmark supports both offline and online RL, and enables direct comparison between discrete-time and continuous-time methods under irregular treatment timing and patient-specific dynamics. Besides, MedGym supports evaluation from clinically important perspectives, including personalization, trajectory-level safety, and the performance gap between model-based offline learning and online deployment. By providing a standardized and configurable benchmark for continuous-time dynamic treatment, MedGym aims to facilitate more realistic and informative evaluation of medical RL methods.