Cvar deep learning improves risk-aware decision timing in finance

CVaR Deep Optimal Stopping

Machine Learning

Summary

Choosing the best time to stop an action can be risky if a bad choice has high costs, like in financial options or medical treatments. The authors study a way to focus on the worst-case outcomes using a measure called Conditional Value at Risk (CVaR). They designed a new method that learns how to stop optimally by fixing key problems in earlier techniques. Their method, called DIOS, learns faster and achieves better risk-aware decisions than previous methods on financial option tests.

What this means in practice

  • For financial engineers: Improve risk-aware timing decisions for exercising financial options using faster, more accurate deep learning methods.
  • For medical treatment planners: Enhance the timing of sequential medical interventions to reduce severe negative outcomes by focusing on worst-case scenarios.

Authors

Yechan Jeong, Youngjoon Hong

Abstract

Risk-sensitive optimal stopping is useful when poor stopping outcomes carry substantial costs, as in option exercise, sequential medical treatment, and safety-critical control. A natural objective is Conditional Value at Risk (CVaR) of the stopped reward. We develop a theoretical framework and algorithm for CVaR deep optimal stopping. The framework and algorithm address four difficulties in applying risk-sensitive reinforcement learning to optimal stopping: state augmentation, score-function pathology, blindness to success, and dependence on auxiliary networks. We isolate three structural properties of optimal stopping: single-point reward, action-independent dynamics, and sum-zero gradient. We exploit these properties to establish Markov optimality without state augmentation and derive a pathwise gradient estimator. Under bounded policy derivatives, the pathwise estimator has bounded second moments near deterministic policies, avoiding the divergence that can occur with score-function estimators. Building on this framework, we propose DIOS (Distribution-Informed Optimal Stopping) to address the remaining two difficulties. A per-time-step soft penalty combined with tail-level annealing mitigates blindness to success. Estimating the reward quantile directly from each batch removes the need for an auxiliary critic. For a fixed tail level, we establish a finite-time stationarity guarantee under regularity assumptions along the training iterates. Across the 24 main Bermudan max-call configurations, DIOS achieves the highest mean CVaR and the shortest recorded training time among the tested CVaR baselines.