Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-sum games of nonlinear stiff systems

In this paper, we establish that for a wide class of controlled stochastic differential equations (SDEs) with stiff coefficients, the value functions of corresponding zero-sum games can be represented by a deep artificial neural network (DNN), whose complexity grows at most polynomially in both the dimension of the state equation and the reciprocal of the required accuracy. Such nonlinear stiff systems may arise, for example, from Galerkin approximations of controlled stochastic partial differential equations (SPDEs), or controlled PDEs with uncertain initial conditions and source terms. This implies that DNNs can break the curse of dimensionality in numerical approximations and optimal control of PDEs and SPDEs. The main ingredient of our proof is to construct a suitable discrete-time system to effectively approximate the evolution of the underlying stochastic dynamics. Similar ideas can also be applied to obtain expression rates of DNNs for value functions induced by stiff systems with regime switching coefficients and driven by general Levy noise.

[1]  Helmut Bölcskei,et al.  Deep Neural Network Approximation Theory , 2019, IEEE Transactions on Information Theory.

[2]  Huyên Pham,et al.  Some machine learning schemes for high-dimensional nonlinear PDEs , 2019, ArXiv.

[3]  Christoph Schwab,et al.  Deep learning in high dimension: Neural network expression rates for generalized polynomial chaos expansions in UQ , 2018, Analysis and Applications.

[4]  Raman Arora,et al.  Understanding Deep Neural Networks with Rectified Linear Units , 2016, Electron. Colloquium Comput. Complex..

[5]  G. Yin,et al.  Hybrid Switching Diffusions , 2010 .

[6]  Annie Millet,et al.  Rate of Convergence of Space Time Approximations for Stochastic Evolution Equations , 2007, 0706.1404.

[7]  Huai-Ning Wu,et al.  Approximate Optimal Control Design for Nonlinear One-Dimensional Parabolic PDE Systems Using Empirical Eigenfunctions and Neural Network , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[8]  Harold J. Kushner,et al.  Optimal stochastic control , 1962 .

[9]  E Weinan,et al.  Exponential convergence of the deep neural network approximation for analytic functions , 2018, Science China Mathematics.

[10]  Jinchao Xu,et al.  Relu Deep Neural Networks and Linear Finite Elements , 2018, Journal of Computational Mathematics.

[11]  Tuan Anh Nguyen,et al.  A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations , 2019, SN Partial Differential Equations and Applications.

[12]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[13]  Shi Jin,et al.  A Stochastic Galerkin Method for Hamilton-Jacobi Equations with Uncertainty , 2015, SIAM J. Sci. Comput..

[14]  E. Jakobsen,et al.  CONTINUOUS DEPENDENCE ESTIMATES FOR VISCOSITY SOLUTIONS OF INTEGRO-PDES , 2005 .

[15]  T. Poggio,et al.  Deep vs. shallow networks : An approximation theory perspective , 2016, ArXiv.

[16]  Fredi Tröltzsch,et al.  ERROR ESTIMATES FOR THE FINITE ELEMENT APPROXIMATION OF A SEMILINEAR ELLIPTIC CONTROL PROBLEM WITH STATE CONSTRAINTS AND FINITE DIMENSIONAL CONTROL SPACE , 2010 .

[17]  E Weinan,et al.  Machine Learning Approximation Algorithms for High-Dimensional Fully Nonlinear Partial Differential Equations and Second-order Backward Stochastic Differential Equations , 2017, J. Nonlinear Sci..

[18]  C. Reisinger,et al.  Stochastic Finite Differences and Multilevel Monte Carlo for a Class of SPDEs in Finance , 2012, SIAM J. Financial Math..

[19]  Dmitry Yarotsky,et al.  Error bounds for approximations with deep ReLU networks , 2016, Neural Networks.

[20]  Qiang Du,et al.  New error bounds for deep networks using sparse grids. , 2017, 1712.08688.

[21]  Gitta Kutyniok,et al.  Error bounds for approximations with deep ReLU neural networks in $W^{s, p}$ norms , 2019, Analysis and Applications.

[22]  K. Ito.  Approximation of the Zakai Equation for Nonlinear Filtering , 1996 .

[23]  É. Pardoux.  Backward Stochastic Differential Equations and Viscosity Solutions of Systems of Semilinear Parabolic and Elliptic PDEs of Second Order , 1998 .

[24]  É. Pardoux.  BSDEs, weak convergence and homogenization of semilinear PDEs , 1999 .

[25]  Fredi Tröltzsch,et al.  Error Estimates for the Finite Element Discretization of Semi-infinite Elliptic Optimal Control Problems , 2010 .

[26]  P. Kloeden,et al.  CONVERGENCE AND STABILITY OF IMPLICIT METHODS FOR JUMP-DIFFUSION SYSTEMS , 2005 .

[27]  Hyunjoong Kim,et al.  Functional Analysis I , 2017 .

[28]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[29]  Claude Jeffrey Gittelson,et al.  Sparse tensor discretizations of high-dimensional parametric and stochastic PDEs* , 2011, Acta Numerica.

[30]  H. Bungartz,et al.  Sparse grids , 2004, Acta Numerica.

[31]  G. Yin,et al.  Hybrid Switching Diffusions: Properties and Applications , 2009 .

[32]  Helmut Bölcskei,et al.  Optimal Approximation with Sparsely Connected Deep Neural Networks , 2017, SIAM J. Math. Data Sci..

[33]  Michael B. Giles,et al.  Adaptive Euler-Maruyama Method for SDEs with Non-globally Lipschitz Drift: Part I, Finite Time Interval , 2016, 1703.06743.

[34]  Karl Kunisch,et al.  Polynomial Approximation of High-Dimensional Hamilton-Jacobi-Bellman Equations and Applications to Feedback Control of Semilinear Parabolic PDEs , 2017, SIAM J. Sci. Comput..

[35]  H. Brezis.  Functional Analysis, Sobolev Spaces and Partial Differential Equations , 2010 .

[36]  Geir E. Dullerud,et al.  Control of Systems With Uncertain Initial Conditions , 2008, IEEE Transactions on Automatic Control.

[37]  Kazufumi Ito,et al.  A neural network based policy iteration algorithm with global H2-superlinear convergence for stochastic games on domains , 2019, Found. Comput. Math..

[38]  Arnulf Jentzen,et al.  Overcoming the Curse of Dimensionality in the Numerical Approximation of Parabolic Partial Differential Equations with Gradient-Dependent Nonlinearities , 2019, Foundations of Computational Mathematics.

[39]  Arnulf Jentzen,et al.  DNN Expression Rate Analysis of High-Dimensional PDEs: Application to Option Pricing , 2018, Constructive Approximation.

[40]  P. Kloeden,et al.  Strong convergence of an explicit numerical method for SDEs with nonglobally Lipschitz continuous coefficients , 2010, 1010.3756.

[41]  Arnulf Jentzen,et al.  A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients , 2018, Communications in Mathematical Sciences.

[42]  Christoph Reisinger,et al.  Stochastic Evolution Equations in Portfolio Credit Modelling , 2011, SIAM J. Financial Math..

[43]  Christoph Schwab,et al.  Deep ReLU networks and high-order finite element methods , 2020, Analysis and Applications.

[44]  A. Barth,et al.  Simulation of stochastic partial differential equations using finite element methods , 2012 .

[45]  Han-Xiong Li,et al.  Adaptive Optimal Control of Highly Dissipative Nonlinear Spatially Distributed Processes With Neuro-Dynamic Programming , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[46]  Qiang Du,et al.  New Error Bounds for Deep ReLU Networks Using Sparse Grids , 2017, SIAM J. Math. Data Sci..

[47]  Jessica Fuerst,et al.  Stochastic Differential Equations And Applications , 2016 .

[48]  P. Kloeden,et al.  Taylor Approximations for Stochastic Partial Differential Equations , 2011 .

[49]  Philipp Petersen,et al.  Optimal approximation of piecewise smooth functions using deep ReLU neural networks , 2017, Neural Networks.

[50]  Huyen Pham,et al.  Neural networks-based backward scheme for fully nonlinear PDEs , 2019, SN Partial Differential Equations and Applications.

[51]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[52]  Xuerong Mao,et al.  Strong convergence and stability of implicit numerical methods for stochastic differential equations with non-globally Lipschitz continuous coefficients , 2012, J. Comput. Appl. Math..

[53]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[54]  Lorenzo Rosasco,et al.  Why and when can deep-but not shallow-networks avoid the curse of dimensionality: A review , 2016, International Journal of Automation and Computing.

[55]  T. Kurtz,et al.  Particle representations for a class of nonlinear SPDEs , 1999 .