Data-Driven Deep Learning of Partial Differential Equations in Modal Space

Abstract We present a framework for recovering/approximating unknown time-dependent partial differential equation (PDE) using its solution data. Instead of identifying the terms in the underlying PDE, we seek to approximate the evolution operator of the underlying PDE numerically. The evolution operator of the PDE, defined in infinite-dimensional space, maps the solution from a current time to a future time and completely characterizes the solution evolution of the underlying unknown PDE. Our recovery strategy relies on approximation of the evolution operator in a properly defined modal space, i.e., generalized Fourier space, in order to reduce the problem to finite dimensions. The finite dimensional approximation is then accomplished by training a deep neural network structure, which is based on residual network (ResNet), using the given data. Error analysis is provided to illustrate the predictive accuracy of the proposed method. A set of examples of different types of PDEs, including inviscid Burgers' equation that develops discontinuity in its solution, are presented to demonstrate the effectiveness of the proposed method.

[1]  Ahmed H. Elsheikh,et al.  A machine learning approach for efficient uncertainty quantification using multiscale methods , 2017, J. Comput. Phys..

[2]  Giang Tran,et al.  Exact Recovery of Chaotic Systems from Highly Corrupted Data , 2016, Multiscale Model. Simul..

[3]  Nicholas Zabaras,et al.  Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification , 2018, J. Comput. Phys..

[4]  Bin Dong,et al.  PDE-Net: Learning PDEs from Data , 2017, ICML.

[5]  G. Karniadakis,et al.  Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems , 2018, 1801.01236.

[6]  S. Pawar,et al.  A deep learning enabler for nonintrusive reduced order modeling of fluid flows , 2019, Physics of Fluids.

[7]  Paris Perdikaris,et al.  Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations , 2017, ArXiv.

[8]  Paris Perdikaris,et al.  Machine learning of linear differential equations using Gaussian processes , 2017, J. Comput. Phys..

[9]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[10]  Bin Dong,et al.  PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network , 2018, J. Comput. Phys..

[11]  Steven L. Brunton,et al.  Deep learning of dynamics and signal-noise decomposition with time-stepping constraints , 2018, J. Comput. Phys..

[12]  Hod Lipson,et al.  Distilling Free-Form Natural Laws from Experimental Data , 2009, Science.

[13]  Kailiang Wu,et al.  Data Driven Governing Equations Approximation Using Deep Neural Networks , 2018, J. Comput. Phys..

[14]  Jan S. Hesthaven,et al.  An artificial neural network as a troubled-cell indicator , 2018, J. Comput. Phys..

[15]  Kailiang Wu,et al.  Numerical Aspects for Approximating Governing Equations Using Data , 2018, J. Comput. Phys..

[16]  Hod Lipson,et al.  Automated reverse engineering of nonlinear dynamical systems , 2007, Proceedings of the National Academy of Sciences.

[17]  Steven L. Brunton,et al.  Chaos as an intermittently forced linear system , 2016, Nature Communications.

[18]  Paris Perdikaris,et al.  Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations , 2017, ArXiv.

[19]  Lexing Ying,et al.  Solving parametric PDE problems with artificial neural networks , 2017, European Journal of Applied Mathematics.

[20]  E Weinan,et al.  Solving many-electron Schrödinger equation using deep neural networks , 2018, J. Comput. Phys..

[21]  Ronan Fablet,et al.  EM-like Learning Chaotic Dynamics from Noisy and Partial Observations , 2019, ArXiv.

[22]  George E. Karniadakis,et al.  Hidden physics models: Machine learning of nonlinear partial differential equations , 2017, J. Comput. Phys..

[23]  Jitesh H. Panchal,et al.  Simulator-free solution of high-dimensional stochastic elliptic partial differential equations using deep neural networks , 2019, J. Comput. Phys..

[24]  H. Schaeffer,et al.  Learning partial differential equations via data discovery and sparse optimization , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[25]  Ilias Bilionis,et al.  Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification , 2018, J. Comput. Phys..

[26]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[27]  Maziar Raissi,et al.  Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations , 2018, J. Mach. Learn. Res..

[28]  J. Hesthaven,et al.  Non-intrusive reduced order modeling of nonlinear problems using neural networks , 2018, J. Comput. Phys..

[29]  Guang Lin,et al.  Efficient Deep Learning Techniques for Multiphase Flow Simulation in Heterogeneous Porous Media , 2019, J. Comput. Phys..

[30]  J N Kutz,et al.  Model selection for dynamical systems via sparse regression and information criteria , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[31]  Hayden Schaeffer,et al.  Extracting Sparse High-Dimensional Dynamics from Limited Data , 2017, SIAM J. Appl. Math..

[32]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Arnulf Jentzen,et al.  Solving high-dimensional partial differential equations using deep learning , 2017, Proceedings of the National Academy of Sciences.

[34]  Steven L. Brunton,et al.  Data-driven discovery of partial differential equations , 2016, Science Advances.

[35]  Kailiang Wu,et al.  Structure-preserving Method for Reconstructing Unknown Hamiltonian Systems from Trajectory Data , 2019, SIAM J. Sci. Comput..

[36]  Bin Dong,et al.  Learning to Discretize: Solving 1D Scalar Conservation Laws via Deep Reinforcement Learning , 2019, Communications in Computational Physics.

[37]  Hayden Schaeffer,et al.  Sparse model selection via integral terms. , 2017, Physical review. E.

[38]  Yifan Sun,et al.  NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data , 2019, MSML.

[39]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[40]  Yingjie Liu,et al.  IDENT: Identifying Differential Equations with Numerical Time Evolution , 2019, Journal of Scientific Computing.