Deep learning method improves control of systems with random jumps
Deep Pontryagin Learning for Continuous-Time Stochastic Control with Jumps
Machine Learning
Summary
Controlling complex systems that can suddenly change in unpredictable ways is hard, especially when these changes happen over continuous time. The authors introduce a deep learning framework that separates understanding how small changes affect the system from improving control decisions. Their method uses mathematical tools called the Pontryagin principle to guide learning more efficiently. Tests with simulations show this approach is accurate, stable, and works well for large problems.
What this means in practice
- •For financial engineers: Optimize trading strategies in markets where asset prices can jump suddenly by integrating efficient state sensitivity learning.
- •For robotics control engineers: Improve control algorithms for robots operating in environments with abrupt changes or shocks using deep Pontryagin learning.
Authors
Jean-Loup Dupret, Patrick Cheridito
Abstract
We develop a deep Pontryagin learning framework for high-dimensional continuous-time stochastic control problems that naturally accommodates jump-diffusions. In contrast to direct policy optimization, which repeatedly differentiates the objective through the entire time-discretized dynamics, our method separates adjoint learning from policy improvement by explicitly incorporating state-sensitivity information from the Pontryagin stochastic maximum principle. For a fixed Markov control, we first compute pathwise adjoint labels by backpropagating the reward through time along simulated state trajectories, and project these labels onto the Markov state through a supervised regression step. The control is then updated by enforcing the pointwise Hamiltonian stationarity condition. We prove that the resulting projected adjoint process satisfies a discrete-time Pontryagin principle and converges, as the time step tends to zero, to the solution of the corresponding continuous-time adjoint backward stochastic differential equation. Numerical experiments illustrate the accuracy, stability, and scalability of the approach.