Weak Adversarial Networks for High-dimensional Partial Differential Equations

Abstract Solving general high-dimensional partial differential equations (PDE) is a long-standing challenge in numerical mathematics. In this paper, we propose a novel approach to solve high-dimensional linear and nonlinear PDEs defined on arbitrary domains by leveraging their weak formulations. We convert the problem of finding the weak solution of PDEs into an operator norm minimization problem induced from the weak formulation. The weak solution and the test function in the weak formulation are then parameterized as the primal and adversarial networks respectively, which are alternately updated to approximate the optimal network parameter setting. Our approach, termed as the weak adversarial network (WAN), is fast, stable, and completely mesh-free, which is particularly suitable for high-dimensional PDEs defined on irregular domains where the classical numerical methods based on finite differences and finite elements suffer the issues of slow computation, instability and the curse of dimensionality. We apply our method to a variety of test problems with high-dimensional PDEs to demonstrate its promising performance.

[1]  Silvia Ferrari,et al.  A constrained integration (CINT) approach to solving partial differential equations using artificial neural networks , 2015, Neurocomputing.

[2]  Paris Perdikaris,et al.  Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations , 2017, ArXiv.

[3]  Hadi Meidani,et al.  A deep learning solution approach for high-dimensional random differential equations , 2019, Probabilistic Engineering Mechanics.

[4]  E Weinan,et al.  Overcoming the curse of dimensionality: Solving high-dimensional partial differential equations using deep learning , 2017, ArXiv.

[5]  J. Crank,et al.  A practical method for numerical evaluation of solutions of partial differential equations of the heat-conduction type , 1947 .

[6]  A. Quarteroni,et al.  Numerical Approximation of Partial Differential Equations , 2008 .

[7]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[8]  Kailai Xu,et al.  Deep Learning for Partial Differential Equations ( PDEs ) , 2018 .

[9]  Paris Perdikaris,et al.  Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations , 2017, ArXiv.

[10]  Yoshiro Suzuki,et al.  Neural network-based discretization of nonlinear differential equations , 2019, Neural Computing and Applications.

[11]  E Weinan,et al.  Deep Learning-Based Numerical Methods for High-Dimensional Parabolic Partial Differential Equations and Backward Stochastic Differential Equations , 2017, Communications in Mathematics and Statistics.

[12]  Andrew J. Meade,et al.  The numerical solution of linear ordinary differential equations by feedforward neural networks , 1994 .

[13]  Naif Alajlan,et al.  Artificial Neural Network Methods for the Solution of Second Order Boundary Value Problems , 2019, Computers, Materials & Continua.

[14]  N. Phan-Thien,et al.  Neural-network-based approximations for solving partial differential equations , 1994 .

[16]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[17]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[18]  Hyuk Lee,et al.  Neural algorithm for solving differential equations , 1990 .

[19]  Kaj Nyström,et al.  A unified deep artificial neural network approach to partial differential equations in complex geometries , 2017, Neurocomputing.

[20]  Mona E. Zaghloul,et al.  Analog cellular neural network with application to partial differential equations with variable mesh-size , 1994, Proceedings of IEEE International Symposium on Circuits and Systems - ISCAS '94.

[21]  Paris Perdikaris,et al.  Adversarial Uncertainty Quantification in Physics-Informed Neural Networks , 2018, J. Comput. Phys..

[22]  Ken Perlin,et al.  Accelerating Eulerian Fluid Simulation With Convolutional Networks , 2016, ICML.

[23]  Justin A. Sirignano,et al.  DGM: A deep learning algorithm for solving partial differential equations , 2017, J. Comput. Phys..

[24]  Marcel Bauer,et al.  Numerical Methods for Partial Differential Equations , 1994 .

[25]  E Weinan,et al.  The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems , 2017, Communications in Mathematics and Statistics.

[26]  Randall J. LeVeque,et al.  Finite difference methods for ordinary and partial differential equations - steady-state and time-dependent problems , 2007 .

[27]  Andrew J. Meade,et al.  Solution of nonlinear ordinary differential equations by feedforward neural networks , 1994 .

[28]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[29]  Lexing Ying,et al.  Solving for high-dimensional committor functions using artificial neural networks , 2018, Research in the Mathematical Sciences.

[30]  Arnulf Jentzen,et al.  Solving high-dimensional partial differential equations using deep learning , 2017, Proceedings of the National Academy of Sciences.

[31]  P. Bassanini,et al.  Elliptic Partial Differential Equations of Second Order , 1997 .

[32]  Mona E. Zaghloul,et al.  VLSI implementation of locally connected neural network for solving partial differential equations , 1996 .

[33]  Martin Magill,et al.  Neural Networks Trained to Solve Differential Equations Learn General Representations , 2018, NeurIPS.

[34]  Hadi Meidani,et al.  A Deep Neural Network Surrogate for High-Dimensional Random Partial Differential Equations , 2018, Probabilistic Engineering Mechanics.

[35]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[36]  Akihiko Takahashi,et al.  Asymptotic Expansion as Prior Knowledge in Deep Learning Method for High dimensional BSDEs , 2017, Asia-Pacific Financial Markets.

[37]  E Weinan,et al.  Machine Learning Approximation Algorithms for High-Dimensional Fully Nonlinear Partial Differential Equations and Second-order Backward Stochastic Differential Equations , 2017, J. Nonlinear Sci..

[38]  Jerry M. Mendel,et al.  Structured trainable networks for matrix algebra , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[39]  J. W. Thomas.  Numerical Partial Differential Equations: Finite Difference Methods , 1995 .

[40]  I. Gladwell,et al.  A Survey of Numerical Methods for Partial Differential Equations , 2021, An Introduction to Numerical Methods and Analysis.