Data-driven discovery of free-form governing differential equations

We present a method of discovering governing differential equations from data without the need to specify a priori the terms to appear in the equation. The input to our method is a dataset (or ensemble of datasets) corresponding to a particular solution (or ensemble of particular solutions) of a differential equation. The output is a human-readable differential equation with parameters calibrated to the individual particular solutions provided. The key to our method is to learn differentiable models of the data that subsequently serve as inputs to a genetic programming algorithm in which graphs specify computation over arbitrary compositions of functions, parameters, and (potentially differential) operators on functions. Differential operators are composed and evaluated using recursive application of automatic differentiation, allowing our algorithm to explore arbitrary compositions of operators without the need for human intervention. We also demonstrate an active learning process to identify and remedy deficiencies in the proposed governing equations.

[1]  Paris Perdikaris,et al.  Machine learning of linear differential equations using Gaussian processes , 2017, J. Comput. Phys..

[2]  Hod Lipson,et al.  Distilling Free-Form Natural Laws from Experimental Data , 2009, Science.

[3]  Anders Logg,et al.  Automated Solution of Differential Equations by the Finite Element Method: The FEniCS Book , 2012 .

[4]  Bin Dong,et al.  PDE-Net: Learning PDEs from Data , 2017, ICML.

[5]  Marc Parizeau,et al.  DEAP: evolutionary algorithms made easy , 2012, J. Mach. Learn. Res..

[6]  G. Karniadakis,et al.  Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems , 2018, 1801.01236.

[7]  Paris Perdikaris,et al.  Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations , 2017, ArXiv.

[8]  Hod Lipson,et al.  Learning symbolic representations of hybrid dynamical systems , 2012, J. Mach. Learn. Res..

[9]  Maziar Raissi,et al.  Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations , 2018, J. Mach. Learn. Res..

[10]  Marco Aurélio Cavalcanti Pacheco,et al.  Solving stochastic differential equations through genetic programming and automatic differentiation , 2018, Eng. Appl. Artif. Intell..

[11]  Paris Perdikaris,et al.  Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations , 2017, ArXiv.

[12]  Isaac E. Lagaris,et al.  Solving differential equations with genetic programming , 2006, Genetic Programming and Evolvable Machines.

[13]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[14]  Dario Izzo,et al.  Differentiable Genetic Programming , 2016, EuroGP.

[15]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[16]  Sean Gerrish,et al.  Black Box Variational Inference , 2013, AISTATS.

[17]  Constantinos I. Siettos,et al.  Coarse-scale PDEs from fine-scale observations via machine learning , 2019, Chaos.

[18]  Nicholas Zabaras,et al.  Structured Bayesian Gaussian process latent variable model: applications to data-driven dimensionality reduction and high-dimensional inversion , 2019, J. Comput. Phys..

[19]  Hod Lipson,et al.  The evolutionary origins of modularity , 2012, Proceedings of the Royal Society B: Biological Sciences.

[20]  Steven L. Brunton,et al.  Data-Driven Identification of Parametric Partial Differential Equations , 2018, SIAM J. Appl. Dyn. Syst..

[21]  George E. Karniadakis,et al.  Hidden physics models: Machine learning of nonlinear partial differential equations , 2017, J. Comput. Phys..

[22]  H. Schaeffer,et al.  Learning partial differential equations via data discovery and sparse optimization , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[23]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[24]  Hod Lipson,et al.  Automated reverse engineering of nonlinear dynamical systems , 2007, Proceedings of the National Academy of Sciences.

[25]  Steven L. Brunton,et al.  Data-driven discovery of partial differential equations , 2016, Science Advances.

[26]  Adam Gaier,et al.  Weight Agnostic Neural Networks , 2019, NeurIPS.