Overcoming the Curse of Dimensionality in the Numerical Approximation of Parabolic Partial Differential Equations with Gradient-Dependent Nonlinearities

Partial differential equations (PDEs) are a fundamental tool in the modeling of many real world phenomena. In a number of such real world phenomena the PDEs under consideration contain gradient-dependent nonlinearities and are high-dimensional. Such high-dimensional nonlinear PDEs can in nearly all cases not be solved explicitly and it is one of the most challenging tasks in applied mathematics to solve high-dimensional nonlinear PDEs approximately. It is especially very challenging to design approximation algorithms for nonlinear PDEs for which one can rigorously prove that they do overcome the so-called curse of dimensionality in the sense that the number of computational operations of the approximation algorithm needed to achieve an approximation precision of size $\varepsilon$ > 0 grows at most polynomially in both the PDE dimension $d \in \mathbb{N}$ and the reciprocal of the prescribed approximation accuracy $\varepsilon$. In particular, to the best of our knowledge there exists no approximation algorithm in the scientific literature which has been proven to overcome the curse of dimensionality in the case of a class of nonlinear PDEs with general time horizons and gradient-dependent nonlinearities. It is the key contribution of this article to overcome this difficulty. More specifically, it is the key contribution of this article (i) to propose a new full-history recursive multilevel Picard approximation algorithm for high-dimensional nonlinear heat equations with general time horizons and gradient-dependent nonlinearities and (ii) to rigorously prove that this full-history recursive multilevel Picard approximation algorithm does indeed overcome the curse of dimensionality in the case of such nonlinear heat equations with gradient-dependent nonlinearities.

[1]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[2]  E Weinan,et al.  The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems , 2017, Communications in Mathematics and Statistics.

[3]  Andrew J. Meade,et al.  The numerical solution of linear ordinary differential equations by feedforward neural networks , 1994 .

[4]  Deep Ray,et al.  Deep learning observables in computational fluid dynamics , 2019, J. Comput. Phys..

[5]  E. Gobet,et al.  A regression-based Monte Carlo method to solve backward stochastic differential equations , 2005, math/0508491.

[6]  Feng Qi (祁锋) Bounds for the Ratio of Two Gamma Functions , 2009 .

[7]  Arnulf Jentzen,et al.  Analysis of the generalization error: Empirical risk minimization over deep artificial neural networks overcomes the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations , 2018, SIAM J. Math. Data Sci..

[8]  Stefan Heinrich,et al.  Monte Carlo Complexity of Parametric Integration , 1999, J. Complex..

[9]  Antoine Jacquier,et al.  Deep PPDEs for Rough Local Stochastic Volatility , 2019, SSRN Electronic Journal.

[10]  Emmanuel Gobet,et al.  Approximation of backward stochastic differential equations using Malliavin weights and least-squares regression , 2016, 1601.01186.

[11]  Pierre Henry-Labordere,et al.  Counterparty Risk Valuation: A Marked Branching Diffusion Approach , 2012, 1203.2369.

[12]  Arnulf Jentzen,et al.  Multilevel Picard iterations for solving smooth semilinear parabolic heat equations , 2021, Partial Differential Equations and Applications.

[13]  Daniel Nikovski,et al.  Deep reinforcement learning for partial differential equation control , 2017, 2017 American Control Conference (ACC).

[14]  Tuan Anh Nguyen,et al.  A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations , 2019, SN Partial Differential Equations and Applications.

[15]  Siwei Luo,et al.  Numerical solution of elliptic partial differential equation using radial basis function neural networks , 2003, Neural Networks.

[16]  B. Bouchard,et al.  Discrete-time approximation and Monte-Carlo simulation of backward stochastic differential equations , 2004 .

[17]  Luo Siwei,et al.  Numerical solution of elliptic partial differential equation using radial basis function neural networks. , 2003, Neural networks : the official journal of the International Neural Network Society.

[18]  Christoph Reisinger,et al.  Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-sum games of nonlinear stiff systems , 2019, Analysis and Applications.

[19]  Michael B. Giles,et al.  Multilevel Monte Carlo Path Simulation , 2008, Oper. Res..

[20]  Nadia Oudjane,et al.  Branching diffusion representation of semilinear PDEs and Monte Carlo approximation , 2016, Annales de l'Institut Henri Poincaré, Probabilités et Statistiques.

[21]  Justin A. Sirignano,et al.  DGM: A deep learning algorithm for solving partial differential equations , 2017, J. Comput. Phys..

[22]  Arnulf Jentzen,et al.  Overcoming the curse of dimensionality in the numerical approximation of Allen–Cahn partial differential equations via truncated full-history recursive multilevel Picard approximations , 2019, J. Num. Math..

[23]  N. Sonehara,et al.  Solving inverse problems in nonlinear PDEs by recurrent neural networks , 1993, IEEE International Conference on Neural Networks.

[24]  Tim Dockhorn,et al.  A Discussion on Solving Partial Differential Equations using Neural Networks , 2019, ArXiv.

[25]  Andrea Molent,et al.  Machine learning for pricing American options in high-dimensional Markovian and non-Markovian models , 2019, Quantitative Finance.

[26]  Arnulf Jentzen,et al.  Numerical simulations for full history recursive multilevel Picard approximations for systems of high-dimensional partial differential equations , 2020, ArXiv.

[27]  Arnulf Jentzen,et al.  Space-time error estimates for deep neural network approximations for differential equations , 2019, Advances in Computational Mathematics.

[28]  Sebastian Becker,et al.  Solving stochastic differential equations and Kolmogorov equations by means of deep learning , 2018, ArXiv.

[29]  Arnulf Jentzen,et al.  Generalised multilevel Picard approximations , 2019, ArXiv.

[30]  Gitta Kutyniok,et al.  A Theoretical Analysis of Deep Neural Networks and Parametric PDEs , 2019, Constructive Approximation.

[31]  E Weinan,et al.  Deep Learning-Based Numerical Methods for High-Dimensional Parabolic Partial Differential Equations and Backward Stochastic Differential Equations , 2017, Communications in Mathematics and Statistics.

[32]  Arnulf Jentzen,et al.  Deep neural network approximations for Monte Carlo algorithms , 2019, ArXiv.

[33]  Arnulf Jentzen,et al.  A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients , 2018, Communications in Mathematical Sciences.

[34]  Kaj Nyström,et al.  A unified deep artificial neural network approach to partial differential equations in complex geometries , 2017, Neurocomputing.

[35]  Sebastian Becker,et al.  Deep Optimal Stopping , 2018, J. Mach. Learn. Res..

[36]  Martin Magill,et al.  Neural Networks Trained to Solve Differential Equations Learn General Representations , 2018, NeurIPS.

[37]  Arnulf Jentzen,et al.  Deep splitting method for parabolic PDEs , 2019, SIAM J. Sci. Comput..

[38]  Philippe von Wurstemberger,et al.  Overcoming the curse of dimensionality in the approximative pricing of financial derivatives with default risks , 2019, Electronic Journal of Probability.

[39]  S. Peng,et al.  Backward Stochastic Differential Equations in Finance , 1997 .

[40]  P. Briand,et al.  Simulation of BSDEs by Wiener Chaos Expansion , 2012, 1204.4137.

[41]  Jiequn Han,et al.  Convergence of the deep BSDE method for coupled FBSDEs , 2018, Probability, Uncertainty and Quantitative Risk.

[42]  Sebastian Becker,et al.  Solving high-dimensional optimal stopping problems using deep learning , 2019, European Journal of Applied Mathematics.

[43]  E Weinan,et al.  Machine Learning Approximation Algorithms for High-Dimensional Fully Nonlinear Partial Differential Equations and Second-order Backward Stochastic Differential Equations , 2017, J. Nonlinear Sci..

[44]  Akihiko Takahashi,et al.  Asymptotic Expansion as Prior Knowledge in Deep Learning Method for High dimensional BSDEs , 2017, Asia-Pacific Financial Markets.

[45]  Bin Dong,et al.  PDE-Net: Learning PDEs from Data , 2017, ICML.

[46]  Justin W. L. Wan,et al.  Deep neural network framework based on backward stochastic differential equations for pricing and hedging American options in high dimensions , 2019, Quantitative Finance.

[47]  Stefan Heinrich,et al.  Monte Carlo Complexity of Global Solution of Integral Equations , 1998, J. Complex..

[48]  Thomas Kruse,et al.  Multilevel Picard Approximations of High-Dimensional Semilinear Parabolic Differential Equations with Gradient-Dependent Nonlinearities , 2017, SIAM J. Numer. Anal..

[49]  Bruno Bouchard,et al.  Numerical approximation of BSDEs using local polynomial drivers and branching processes , 2017, Monte Carlo Methods Appl..

[50]  Xavier Warin,et al.  Machine Learning for Semi Linear PDEs , 2018, Journal of Scientific Computing.

[51]  E. Gobet,et al.  Rate of convergence of an empirical regression method for solving generalized backward stochastic differential equations , 2006 .

[52]  Christel Geiss,et al.  Simulation of BSDEs with jumps by Wiener Chaos expansion , 2015, 1502.05649.

[53]  Xiaolu Tan,et al.  A numerical algorithm for a class of BSDEs via branching process , 2013 .

[54]  Arnulf Jentzen,et al.  Overcoming the curse of dimensionality in the numerical approximation of semilinear parabolic partial differential equations , 2018, Proceedings of the Royal Society A.

[55]  Arnulf Jentzen,et al.  A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations , 2018, Memoirs of the American Mathematical Society.

[56]  Arnulf Jentzen,et al.  Solving high-dimensional partial differential equations using deep learning , 2017, Proceedings of the National Academy of Sciences.

[57]  Pierre Henry-Labordere,et al.  Deep Primal-Dual Algorithm for BSDEs: Applications of Machine Learning to CVA and IM , 2017 .

[58]  E Weinan,et al.  On Multilevel Picard Numerical Approximations for High-Dimensional Nonlinear Parabolic Partial Differential Equations and High-Dimensional Nonlinear Backward Stochastic Differential Equations , 2017, Journal of Scientific Computing.

[59]  J. G. Wendel Note on the Gamma Function , 1948 .

[60]  Maziar Raissi,et al.  Forward-Backward Stochastic Neural Networks: Deep Learning of High-dimensional Partial Differential Equations , 2018, ArXiv.

[61]  Huyen Pham,et al.  Neural networks-based backward scheme for fully nonlinear PDEs , 2019, SN Partial Differential Equations and Applications.

[62]  Arnulf Jentzen,et al.  DNN Expression Rate Analysis of High-Dimensional PDEs: Application to Option Pricing , 2018, Constructive Approximation.

[63]  Huyên Pham,et al.  Some machine learning schemes for high-dimensional nonlinear PDEs , 2019, ArXiv.