Papers for

medical treatment planners

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Cvar deep learning improves risk-aware decision timing in finance

CVaR Deep Optimal Stopping

Abstract: Risk-sensitive optimal stopping is useful when poor stopping outcomes carry substantial costs, as in option exercise, sequential medical treatment, and safety-critical control. A natural objective is Conditional Value at Risk (CVaR) of the stopped reward. We develop a theoretical framework and algorithm for CVaR deep optimal stopping. The framework and algorithm address four difficulties in applying risk-sensitive reinforcement learning to optimal stopping: state augmentation, score-function pathology, blindness to success, and dependence on auxiliary networks. We isolate three structural properties of optimal stopping: single-point reward, action-independent dynamics, and sum-zero gradient. We exploit these properties to establish Markov optimality without state augmentation and derive a pathwise gradient estimator. Under bounded policy derivatives, the pathwise estimator has bounded second moments near deterministic policies, avoiding the divergence that can occur with score-function estimators. Building on this framework, we propose DIOS (Distribution-Informed Optimal Stopping) to address the remaining two difficulties. A per-time-step soft penalty combined with tail-level annealing mitigates blindness to success. Estimating the reward quantile directly from each batch removes the need for an auxiliary critic. For a fixed tail level, we establish a finite-time stationarity guarantee under regularity assumptions along the training iterates. Across the 24 main Bermudan max-call configurations, DIOS achieves the highest mean CVaR and the shortest recorded training time among the tested CVaR baselines.

Sun 8 NovMachine Learning
The gist
Choosing the best time to stop an action can be risky if a bad choice has high costs, like in financial options or medical treatments. The authors study a way to focus on the worst-case outcomes using a measure called Conditional Value at Risk (CVaR). They designed a new method that learns how to stop optimally by fixing key problems in earlier techniques. Their method, called DIOS, learns faster and achieves better risk-aware decisions than previous methods on financial option tests.
Open → 2611.11332v1