Error Analysis of Deep Ritz Methods for Elliptic Equations

Using deep neural networks to solve PDEs has attracted a lot of attentions recently. However, why the deep learning method works is falling far behind its empirical success. In this paper, we provide a rigorous numerical analysis on deep Ritz method (DRM) [43] for second order elliptic equations with Drichilet, Neumann and Robin boundary condition, respectively. We establish the first nonasymptotic convergence rate in H norm for DRM using deep networks with smooth activation functions including logistic and hyperbolic tangent functions. Our results show how to set the hyper-parameter of depth and width to achieve the desired convergence rate in terms of number of training samples.

[1]  Kamyar Azizzadenesheli,et al.  Multipole Graph Neural Operator for Parametric Partial Differential Equations , 2020, NeurIPS.

[2]  Kamyar Azizzadenesheli,et al.  Neural Operator: Graph Kernel Network for Partial Differential Equations , 2020, ICLR 2020.

[3]  P. Bassanini,et al.  Elliptic Partial Differential Equations of Second Order , 1997 .

[4]  Stefano Ermon,et al.  Learning Neural PDE Solvers with Convergence Guarantees , 2019, ICLR.

[5]  Tao Luo,et al.  Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory , 2020, ArXiv.

[6]  Arnulf Jentzen,et al.  Solving high-dimensional partial differential equations using deep learning , 2017, Proceedings of the National Academy of Sciences.

[7]  Mones Raslan,et al.  Approximation Rates for Neural Networks with Encodable Weights in Smoothness Spaces , 2020, Neural Networks.

[8]  Bertrand Maury.  Numerical Analysis of a Finite Element/Volume Penalty Method , 2009, SIAM J. Numer. Anal..

[9]  Justin A. Sirignano,et al.  DGM: A deep learning algorithm for solving partial differential equations , 2017, J. Comput. Phys..

[10]  L. R. Scott,et al.  The Mathematical Theory of Finite Element Methods , 1994 .

[11]  E Weinan,et al.  The Barron Space and the Flow-Induced Function Spaces for Neural Network Models , 2021, Constructive Approximation.

[12]  G. Burton.  Sobolev Spaces , 2013 .

[13]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[14]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[15]  Jinchao Xu.  The Finite Neuron Method and Convergence Analysis , 2020, Communications in Computational Physics.

[16]  Johannes Müller,et al.  Deep Ritz revisited , 2019, ICLR 2019.

[17]  N. Phan-Thien,et al.  Neural-network-based approximations for solving partial differential equations , 1994 .

[18]  Min Wang,et al.  A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equations , 2021, ArXiv.

[19]  R. Molinaro,et al.  Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs , 2020, ArXiv.

[20]  Jinchao Xu,et al.  Rademacher Complexity and Numerical Quadrature Analysis of Stable Neural Networks with Applications to Numerical PDEs , 2021, ArXiv.

[21]  E. Weinan,et al.  Some observations on partial differential equations in Barron and multi-layer spaces , 2020, ArXiv.

[22]  Julius Berner,et al.  Numerically Solving Parametric Families of High-Dimensional Kolmogorov Partial Differential Equations via Deep Learning , 2020, NeurIPS.

[23]  Kamyar Azizzadenesheli,et al.  Fourier Neural Operator for Parametric Partial Differential Equations , 2021, ICLR.

[24]  Zhiping Mao,et al.  DeepXDE: A Deep Learning Library for Solving Differential Equations , 2019, AAAI Spring Symposium: MLPS.

[25]  Jinchao Xu,et al.  A Priori Analysis of Stable Neural Network Solutions to Numerical PDEs , 2021, 2104.02903.

[26]  Naif Alajlan,et al.  Artificial Neural Network Methods for the Solution of Second Order Boundary Value Problems , 2019, Computers, Materials & Continua.

[27]  Ronen Basri,et al.  Learning to Optimize Multigrid PDE Solvers , 2019, ICML.

[28]  Marius Zeinhofer,et al.  Error Estimates for the Variational Training of Neural Networks with Boundary Penalty , 2021, ArXiv.

[29]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[30]  E Weinan,et al.  The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems , 2017, Communications in Mathematics and Statistics.

[31]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[32]  Yuling Jiao,et al.  Convergence Rate Analysis for Deep Ritz Method , 2021, Communications in Computational Physics.

[33]  I. Babuska.  The Finite Element Method with Penalty , 1973 .

[34]  Bin Dong,et al.  Learning to Discretize: Solving 1D Scalar Conservation Laws via Deep Reinforcement Learning , 2019, Communications in Computational Physics.

[35]  Nils Thuerey,et al.  Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers , 2020, NeurIPS.

[36]  Shai Ben-David,et al.  Understanding Machine Learning: From Theory to Algorithms , 2014 .

[37]  Katsushi Ikeuchi,et al.  Numerical Partial Differential Equations , 2014, Computer Vision, A Reference Guide.

[38]  G. Karniadakis,et al.  Error estimates of residual minimization using neural networks for linear PDEs , 2020, ArXiv.

[39]  Gang Bao,et al.  Weak Adversarial Networks for High-dimensional Partial Differential Equations , 2019, J. Comput. Phys..

[40]  Philippe G. Ciarlet,et al.  The finite element method for elliptic problems , 2002, Classics in applied mathematics.