IDENT: Identifying Differential Equations with Numerical Time Evolution

Identifying unknown differential equations from a given set of discrete time dependent data is a challenging problem. A small amount of noise can make the recovery unstable, and nonlinearity and differential equations with varying coefficients add complexity to the problem. We assume that the governing partial differential equation (PDE) is a linear combination of a subset of a prescribed dictionary containing different differential terms, and the objective of this paper is to find the correct coefficients. We propose a new direction based on the fundamental idea of convergence analysis of numerical PDE schemes. We utilize Lasso for efficiency, and a performance guarantee is established based on an incoherence property. The main contribution is to validate and correct the results by Time Evolution Error (TEE). The new algorithm, called Identifying Differential Equations with Numerical Time evolution (IDENT), is explored for data with non-periodic boundary conditions, noisy data and PDEs with varying coefficients. From the recovery analysis of Lasso, we propose a new definition of Noise-to-Signal ratio, which better represents the level of noise in the case of PDE identification. We systematically analyze the effects of data generations and downsampling, and propose an order preserving denoising method called Least-Squares Moving Average (LSMA), to preprocess the given data. For the identification of PDEs with varying coefficients, we propose to add Base Element Expansion (BEE) to aide the computation. Various numerical experiments from basic tests to noisy data, downsampling effects and varying coefficients are presented.

[1]  Giang Tran,et al.  Exact Recovery of Chaotic Systems from Highly Corrupted Data , 2016, Multiscale Model. Simul..

[2]  J N Kutz,et al.  Model selection for dynamical systems via sparse regression and information criteria , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[3]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[4]  Ming Zhong,et al.  Nonparametric inference of interaction laws in systems of agents from trajectory data , 2018, Proceedings of the National Academy of Sciences.

[5]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[6]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[7]  Wenjing Liao,et al.  Coherence Pattern-Guided Compressive Sensing with Unresolved Grids , 2011, SIAM J. Imaging Sci..

[8]  George E. Karniadakis,et al.  Hidden physics models: Machine learning of nonlinear partial differential equations , 2017, J. Comput. Phys..

[9]  Bin Dong,et al.  PDE-Net: Learning PDEs from Data , 2017, ICML.

[10]  H. Schaeffer,et al.  Learning partial differential equations via data discovery and sparse optimization , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[11]  Lexing Ying,et al.  SwitchNet: a neural network model for forward and inverse scattering problems , 2018, SIAM J. Sci. Comput..

[12]  Maziar Raissi,et al.  Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations , 2018, J. Mach. Learn. Res..

[13]  Guang Lin,et al.  Robust data-driven discovery of governing physical laws with error bars , 2018, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[14]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[15]  Chi-Wang Shu,et al.  Central Discontinuous Galerkin Methods on Overlapping Cells with a Nonoscillatory Hierarchical Reconstruction , 2007, SIAM J. Numer. Anal..

[16]  J. Tropp.  JUST RELAX: CONVEX PROGRAMMING METHODS FOR SUBSET SELECTION AND SPARSE APPROXIMATION , 2004 .

[17]  M. Fornasier,et al.  Inferring Interaction Rules From Observations of Evolutive Systems I: The Variational Approach , 2016, 1602.00342.

[18]  Steven L. Brunton,et al.  Deep learning for universal linear embeddings of nonlinear dynamics , 2017, Nature Communications.

[19]  Chaowei Hu,et al.  No . 98-32 Weighted Essentially Non-Oscillatory Schemes on Triangular Meshes , 1998 .

[20]  Hayden Schaeffer,et al.  Extracting Sparse High-Dimensional Dynamics from Limited Data , 2017, SIAM J. Appl. Math..

[21]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[22]  Paris Perdikaris,et al.  Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations , 2017, ArXiv.

[23]  Steven L. Brunton,et al.  Constrained sparse Galerkin regression , 2016, Journal of Fluid Mechanics.

[24]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[25]  E Kaiser,et al.  Sparse identification of nonlinear dynamics for model predictive control in the low-data limit , 2017, Proceedings of the Royal Society A.

[26]  P. Frederickson,et al.  Higher order solution of the Euler equations on unstructured grids using quadratic reconstruction , 1990 .

[27]  Steven L. Brunton,et al.  Data-driven discovery of partial differential equations , 2016, Science Advances.

[28]  S. Osher,et al.  Uniformly high order accurate essentially non-oscillatory schemes, 111 , 1987 .

[29]  Kailiang Wu,et al.  Data Driven Governing Equations Approximation Using Deep Neural Networks , 2018, J. Comput. Phys..

[30]  Hod Lipson,et al.  Distilling Free-Form Natural Laws from Experimental Data , 2009, Science.

[31]  Jean-Jacques Fuchs,et al.  On sparse representations in arbitrary redundant bases , 2004, IEEE Transactions on Information Theory.

[32]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[33]  Paris Perdikaris,et al.  Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations , 2017, ArXiv.

[34]  S. Osher,et al.  Uniformly high order accuracy essentially non-oscillatory schemes III , 1987 .

[35]  Hod Lipson,et al.  Automated reverse engineering of nonlinear dynamical systems , 2007, Proceedings of the National Academy of Sciences.

[36]  S. Osher,et al.  Sparse dynamics for partial differential equations , 2012, Proceedings of the National Academy of Sciences.