Solving Irregular and Data-enriched Differential Equations using Deep Neural Networks

Recent work has introduced a simple numerical method for solving partial differential equations (PDEs) with deep neural networks (DNNs). This paper reviews and extends the method while applying it to analyze one of the most fundamental features in numerical PDEs and nonlinear analysis: irregular solutions. First, the Sod shock tube solution to compressible Euler equations is discussed, analyzed, and then compared to conventional finite element and finite volume methods. These methods are extended to consider performance improvements and simultaneous parameter space exploration. Next, a shock solution to compressible magnetohydrodynamics (MHD) is solved for, and used in a scenario where experimental data is utilized to enhance a PDE system that is \emph{a priori} insufficient to validate against the observed/experimental data. This is accomplished by enriching the model PDE system with source terms and using supervised training on synthetic experimental data. The resulting DNN framework for PDEs seems to demonstrate almost fantastical ease of system prototyping, natural integration of large data sets (be they synthetic or experimental), all while simultaneously enabling single-pass exploration of the entire parameter space.

[1]  Eric Darve,et al.  The Neural Network Approach to Inverse Problems in Differential Equations , 2019, 1901.07758.

[2]  J. Garnett,et al.  Bounded Analytic Functions , 2006 .

[3]  Thomas Gerstner,et al.  Numerical integration using sparse grids , 2004, Numerical Algorithms.

[4]  P. Lax Hyperbolic Systems of Conservation Laws and the Mathematical Theory of Shock Waves , 1987 .

[5]  Benjamin Peherstorfer,et al.  Survey of multifidelity methods in uncertainty propagation, inference, and optimization , 2018, SIAM Rev..

[6]  G. Sod A survey of several finite difference methods for systems of nonlinear hyperbolic conservation laws , 1978 .

[7]  G. M.,et al.  Partial Differential Equations I , 2023, Applied Mathematical Sciences.

[8]  Kaj Nyström,et al.  A unified deep artificial neural network approach to partial differential equations in complex geometries , 2017, Neurocomputing.

[9]  Michael Dumbser,et al.  Quadrature-free non-oscillatory finite volume schemes on unstructured meshes for nonlinear hyperbolic systems , 2007, J. Comput. Phys..

[10]  A. J. Roberts,et al.  A centre manifold description of containment dispersion in channels with varying flow properties , 1990 .

[11]  R. LeVeque Finite Volume Methods for Hyperbolic Problems: Characteristics and Riemann Problems for Linear Hyperbolic Equations , 2002 .

[12]  H. Schaeffer,et al.  Learning partial differential equations via data discovery and sparse optimization , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[13]  I. Babuska,et al.  Acta Numerica 2003: Survey of meshless and generalized finite element methods: A unified approach , 2003 .

[14]  Sophia Blau,et al.  Analysis Of The Finite Element Method , 2016 .

[15]  Pavel B. Bochev,et al.  Least-Squares Finite Element Methods , 2009, Applied mathematical sciences.

[16]  Todd A. Oliver,et al.  Validating predictions of unobserved quantities , 2014, 1404.7555.

[17]  Myoungkyu Lee,et al.  Experiences Porting Scientific Applications to the Intel (KNL) Xeon Phi Platform , 2017, PEARC.

[18]  Clint Dawson,et al.  Performance Comparison of HPX Versus Traditional Parallelization Strategies for the Discontinuous Galerkin Method , 2019, J. Sci. Comput..

[19]  John Sibert,et al.  AD Model Builder: using automatic differentiation for statistical inference of highly parameterized complex nonlinear models , 2012, Optim. Methods Softw..

[20]  George G. Lorentz,et al.  Constructive Approximation , 1993, Grundlehren der mathematischen Wissenschaften.

[21]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[22]  Justin A. Sirignano,et al.  DGM: A deep learning algorithm for solving partial differential equations , 2017, J. Comput. Phys..

[23]  Xiaolin Li,et al.  Error estimates for the moving least-square approximation and the element-free Galerkin method in n-dimensional spaces , 2016 .

[24]  J. Smoller Shock Waves and Reaction-Diffusion Equations , 1983 .

[25]  Clint Dawson,et al.  Adaptive hierarchic transformations for dynamically p-enriched slope-limiting over discontinuous Galerkin systems of generalized equations , 2010, J. Comput. Phys..

[26]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[27]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[28]  D. Balsara,et al.  A divergence‐free semi‐implicit finite volume scheme for ideal, viscous, and resistive magnetohydrodynamics , 2018, International journal for numerical methods in fluids.

[29]  Arnulf Jentzen,et al.  Solving high-dimensional partial differential equations using deep learning , 2017, Proceedings of the National Academy of Sciences.

[30]  S. R. Brus,et al.  A Comparison of Artificial Viscosity, Limiters, and Filters, for High Order Discontinuous Galerkin Solutions in Nonlinear Settings , 2016, J. Sci. Comput..

[31]  Steven L. Brunton,et al.  Data-driven discovery of partial differential equations , 2016, Science Advances.

[32]  T. Belytschko,et al.  Element‐free Galerkin methods , 1994 .

[33]  Yoshua Bengio,et al.  On the Spectral Bias of Neural Networks , 2018, ICML.

[34]  Bin Dong,et al.  PDE-Net: Learning PDEs from Data , 2017, ICML.

[35]  Clifford H. Thurber,et al.  Parameter estimation and inverse problems , 2005 .

[36]  Maziar Raissi,et al.  Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations , 2018, J. Mach. Learn. Res..

[37]  J. Blondin,et al.  Hydrodynamic instabilities in supernova remnants : self-similar driven waves , 1992 .

[38]  Zhengfu Xu Parametrized maximum principle preserving flux limiters for high order schemes solving hyperbolic conservation laws: one-dimensional scalar problem , 2014, Math. Comput..

[39]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[40]  John N. Shadid,et al.  Local bounds preserving stabilization for continuous Galerkin discretization of hyperbolic systems , 2018, J. Comput. Phys..

[41]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[42]  Steven L. Brunton,et al.  Data-driven discovery of coordinates and governing equations , 2019, Proceedings of the National Academy of Sciences.

[43]  Taku Ohwada,et al.  Management of discontinuous reconstruction in kinetic schemes , 2004 .

[44]  Zhiliang Xu,et al.  Hierarchical reconstruction for spectral volume method on unstructured grids , 2009, J. Comput. Phys..

[45]  G. Carey,et al.  Least-squares finite element methods for compressible Euler equations , 1990 .

[46]  Craig Michoski,et al.  Foundations of the blended isogeometric discontinuous Galerkin (BIDG) method , 2016 .

[47]  Rémi Abgrall,et al.  Discrete equations for physical and numerical compressible multiphase mixtures , 2003 .

[48]  Liwei Wang,et al.  The Expressive Power of Neural Networks: A View from the Width , 2017, NIPS.

[49]  P. Colella,et al.  Local adaptive mesh refinement for shock hydrodynamics , 1989 .

[50]  George Em Karniadakis,et al.  Spectral element‐FCT method for scalar hyperbolic conservation laws , 1992 .

[51]  Kaj Nyström,et al.  Data-driven discovery of PDEs in complex datasets , 2018, J. Comput. Phys..

[52]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[53]  R. LeVeque Wave Propagation Algorithms for Multidimensional Hyperbolic Systems , 1997 .

[54]  C. Domb,et al.  On the susceptibility of a ferromagnetic above the Curie point , 1957, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[55]  Shulin Zhou,et al.  Optimal regularity for the Poisson equation , 2009 .

[56]  Philip Isett,et al.  A Proof of Onsager's Conjecture , 2016, 1608.08301.

[57]  Zhaoyan Zhang,et al.  Theory of shock wave propagation during laser ablation , 2004 .