A control method for solving high-dimensional Hamiltonian systems through deep neural networks

In this paper, we mainly focus on solving high-dimensional stochastic Hamiltonian systems with boundary condition, and propose a novel method from the view of the stochastic control. In order to obtain the approximated solution of the Hamiltonian system, we first introduce a corresponding stochastic optimal control problem such that the Hamiltonian system of control problem is exactly what we need to solve, then develop two different algorithms suitable for different cases of the control problem and approximate the stochastic control via deep neural networks. From the numerical results, comparing with the Deep FBSDE method which was developed previously from the view of solving FBSDEs, the novel algorithms converge faster, which means that they require fewer training steps, and demonstrate more stable convergences for different Hamiltonian systems.

[1]  G. N. Milstein,et al.  Numerical Algorithms for Forward-Backward Stochastic Differential Equations , 2006, SIAM J. Sci. Comput..

[2]  E Weinan,et al.  Deep Learning Approximation for Stochastic Control Problems , 2016, ArXiv.

[3]  J. Yong,et al.  Forward–backward stochastic differential equations with nonsmooth coefficients , 2000 .

[4]  E. Tadmor.  A review of numerical methods for nonlinear partial differential equations , 2012 .

[5]  Shige Peng,et al.  Problem of eigenvalues of stochastic Hamiltonian systems with boundary conditions , 2000 .

[6]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[7]  Jie Shen,et al.  On Numerical Approximations of Forward-Backward Stochastic Differential Equations , 2008, SIAM J. Numer. Anal..

[8]  Agus Sudjianto,et al.  Deep Learning-Based BSDE Solver for Libor Market Model with Applications to Bermudan Swaption Pricing and Hedging , 2018, 1807.06622.

[9]  Huyên Pham,et al.  Deep backward schemes for high-dimensional nonlinear PDEs , 2019 .

[10]  J. Yong,et al.  Solving forward-backward stochastic differential equations explicitly — a four step scheme , 1994 .

[11]  E Weinan,et al.  Deep Learning-Based Numerical Methods for High-Dimensional Parabolic Partial Differential Equations and Backward Stochastic Differential Equations , 2017, Communications in Mathematics and Statistics.

[12]  S. Peng,et al.  Backward Stochastic Differential Equations in Finance , 1997 .

[13]  Arnulf Jentzen,et al.  Solving high-dimensional partial differential equations using deep learning , 2017, Proceedings of the National Academy of Sciences.

[14]  E Weinan,et al.  On Multilevel Picard Numerical Approximations for High-Dimensional Nonlinear Parabolic Partial Differential Equations and High-Dimensional Nonlinear Backward Stochastic Differential Equations , 2017, Journal of Scientific Computing.

[15]  B. Bouchard,et al.  Discrete-time approximation and Monte-Carlo simulation of backward stochastic differential equations , 2004 .

[16]  Weidong Zhao,et al.  Efficient spectral sparse grid approximations for solving multi-dimensional forward backward SDEs , 2016 .

[17]  Romeo Ortega,et al.  Putting energy back in control , 2001 .

[18]  Nathan Srebro,et al.  The Marginal Value of Adaptive Gradient Methods in Machine Learning , 2017, NIPS.

[19]  Arjan van der Schaft,et al.  Interconnection and damping assignment passivity-based control of port-controlled Hamiltonian systems , 2002, Autom..

[20]  Zhen Wu,et al.  On well-posedness of forward-backward SDEs-A unified approach , 2011, 1110.4658.

[21]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[22]  Christian Bender,et al.  Time discretization and Markovian iteration for coupled FBSDEs , 2008, 0801.3203.

[23]  G. Pagès,et al.  A quantization algorithm for solving multidimensional discrete-time optimal stopping problems , 2003 .

[24]  Cornelis W. Oosterlee,et al.  Computational and Applied , 2015 .

[25]  Jiongmin Yong,et al.  Approximate Solvability of Forward—Backward Stochastic Differential Equations , 2002 .

[26]  Maziar Raissi,et al.  Forward-Backward Stochastic Neural Networks: Deep Learning of High-dimensional Partial Differential Equations , 2018, ArXiv.

[27]  Huyen Pham,et al.  Neural networks-based backward scheme for fully nonlinear PDEs , 2019, SN Partial Differential Equations and Applications.

[28]  S. Peng,et al.  Fully Coupled Forward-Backward Stochastic Differential Equations and Applications to Optimal Control , 1999 .

[29]  Yu Fu,et al.  Multistep Schemes for Forward Backward Stochastic Differential Equations with Jumps , 2016, J. Sci. Comput..

[30]  S. Peng,et al.  Solution of forward-backward stochastic differential equations , 1995 .

[31]  Jin Ma,et al.  Solvability of forward-backward SDEs and the nodal set of Hamilton-Jacobi-Bellman Equations , 1993 .

[32]  É. Pardoux,et al.  Forward-backward stochastic differential equations and quasilinear parabolic PDEs , 1999 .

[33]  A. Bensoussan.  Lectures on stochastic control , 1982 .

[34]  Ying Peng,et al.  Three Algorithms for Solving High-Dimensional Fully Coupled FBSDEs Through Deep Learning , 2019, IEEE Intelligent Systems.