DL-PDE: Deep-learning based data-driven discovery of partial differential equations from discrete and noisy data

In recent years, data-driven methods have been developed to learn dynamical systems and partial differential equations (PDE). The goal of such work is discovering unknown physics and the corresponding equations. However, prior to achieving this goal, major challenges remain to be resolved, including learning PDE under noisy data and limited discrete data. To overcome these challenges, in this work, a deep-learning based data-driven method, called DL-PDE, is developed to discover the governing PDEs of underlying physical processes. The DL-PDE method combines deep learning via neural networks and data-driven discovery of PDE via sparse regressions. In the DL-PDE, a neural network is first trained, and then a large amount of meta-data is generated, and the required derivatives are calculated by automatic differentiation. Finally, the form of PDE is discovered by sparse regression. The proposed method is tested with physical processes, governed by groundwater flow equation, convection-diffusion equation, Burgers equation and Korteweg-de Vries (KdV) equation, for proof-of-concept and applications in real-world engineering settings. The proposed method achieves satisfactory results when data are noisy and limited.

[1]  Bin Dong,et al.  PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network , 2018, J. Comput. Phys..

[2]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[3]  Maziar Raissi,et al.  Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations , 2018, J. Mach. Learn. Res..

[4]  Liu Yang,et al.  Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations , 2018, SIAM J. Sci. Comput..

[5]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[6]  Barak A. Pearlmutter,et al.  Automatic differentiation in machine learning: a survey , 2015, J. Mach. Learn. Res..

[7]  Paris Perdikaris,et al.  Machine learning of linear differential equations using Gaussian processes , 2017, J. Comput. Phys..

[8]  Hayden Schaeffer,et al.  Learning Dynamical Systems and Bifurcation via Group Sparsity , 2017, 1709.01558.

[9]  Linan Zhang,et al.  On the Convergence of the SINDy Algorithm , 2018, Multiscale Model. Simul..

[10]  Feliks Nüske,et al.  Sparse learning of stochastic dynamical equations. , 2017, The Journal of chemical physics.

[11]  Steven L. Brunton,et al.  Methods for data-driven multiscale model discovery for materials , 2019, Journal of Physics: Materials.

[12]  E Kaiser,et al.  Sparse identification of nonlinear dynamics for model predictive control in the low-data limit , 2017, Proceedings of the Royal Society A.

[13]  C. Basdevant,et al.  Spectral and finite difference solutions of the Burgers equation , 1986 .

[14]  J N Kutz,et al.  Model selection for dynamical systems via sparse regression and information criteria , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[15]  Hayden Schaeffer,et al.  Extracting Sparse High-Dimensional Dynamics from Limited Data , 2017, SIAM J. Appl. Math..

[16]  J. Carrera,et al.  A stable computation of log‐derivatives from noisy drawdown data , 2017 .

[17]  Steven L. Brunton,et al.  Data-driven discovery of partial differential equations , 2016, Science Advances.

[18]  Hayden Schaeffer,et al.  Sparse model selection via integral terms. , 2017, Physical review. E.

[19]  Steven L. Brunton,et al.  Data-Driven Identification of Parametric Partial Differential Equations , 2018, SIAM J. Appl. Dyn. Syst..

[20]  George Em Karniadakis,et al.  Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems , 2018, J. Comput. Phys..

[21]  Steven L. Brunton,et al.  Inferring Biological Networks by Sparse Identification of Nonlinear Dynamics , 2016, IEEE Transactions on Molecular, Biological and Multi-Scale Communications.

[22]  George Em Karniadakis,et al.  fPINNs: Fractional Physics-Informed Neural Networks , 2018, SIAM J. Sci. Comput..

[23]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[24]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[25]  George E. Karniadakis,et al.  Hidden physics models: Machine learning of nonlinear partial differential equations , 2017, J. Comput. Phys..

[26]  H. Schaeffer,et al.  Learning partial differential equations via data discovery and sparse optimization , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[27]  Kaj Nyström,et al.  Data-driven discovery of PDEs in complex datasets , 2018, J. Comput. Phys..

[28]  Bin Dong,et al.  PDE-Net: Learning PDEs from Data , 2017, ICML.

[29]  F. Jauberteau,et al.  Numerical differentiation with noisy signal , 2009, Appl. Math. Comput..

[30]  Haibin Chang,et al.  Identification of physical processes via combined data-driven and data-assimilation methods , 2018, J. Comput. Phys..

[31]  J. Cullum Numerical Differentiation and Regularization , 1971 .

[32]  Steven L. Brunton,et al.  Data-driven discovery of coordinates and governing equations , 2019, Proceedings of the National Academy of Sciences.

[33]  Haibin Chang,et al.  Machine learning subsurface flow equations from data , 2019, Computational Geosciences.

[34]  Rick Chartrand,et al.  Numerical Differentiation of Noisy, Nonsmooth Data , 2011 .

[35]  Steven L Brunton,et al.  Sparse identification of nonlinear dynamics for rapid model recovery. , 2018, Chaos.

[36]  Guang Lin,et al.  Robust data-driven discovery of governing physical laws with error bars , 2018, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[37]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[38]  Zhiping Mao,et al.  DeepXDE: A Deep Learning Library for Solving Differential Equations , 2019, AAAI Spring Symposium: MLPS.