Coarse-scale PDEs from fine-scale observations via machine learning

Complex spatiotemporal dynamics of physicochemical processes are often modeled at a microscopic level (through, e.g., atomistic, agent-based, or lattice models) based on first principles. Some of these processes can also be successfully modeled at the macroscopic level using, e.g., partial differential equations (PDEs) describing the evolution of the right few macroscopic observables (e.g., concentration and momentum fields). Deriving good macroscopic descriptions (the so-called "closure problem") is often a time-consuming process requiring deep understanding/intuition about the system of interest. Recent developments in data science provide alternative ways to effectively extract/learn accurate macroscopic descriptions approximating the underlying microscopic observations. In this paper, we introduce a data-driven framework for the identification of unavailable coarse-scale PDEs from microscopic observations via machine-learning algorithms. Specifically, using Gaussian processes, artificial neural networks, and/or diffusion maps, the proposed framework uncovers the relation between the relevant macroscopic space fields and their time evolution (the right-hand side of the explicitly unavailable macroscopic PDE). Interestingly, several choices equally representative of the data can be discovered. The framework will be illustrated through the data-driven discovery of macroscopic, concentration-level PDEs resulting from a fine-scale, lattice Boltzmann level model of a reaction/transport process. Once the coarse evolution law is identified, it can be simulated to produce long-term macroscopic predictions. Different features (pros as well as cons) of alternative machine-learning algorithms for performing this task (Gaussian processes and artificial neural networks) are presented and discussed.

[1]  P. Bhatnagar,et al.  A Model for Collision Processes in Gases. I. Small Amplitude Processes in Charged and Neutral One-Component Systems , 1954 .

[2]  Y. Wong,et al.  Differentiable Manifolds , 2009 .

[3]  F. Takens Detecting strange attractors in turbulence , 1981 .

[4]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[5]  Ioannis G. Kevrekidis,et al.  Nonlinear signal processing and system identification: applications to time series from electrochemical reactions , 1990 .

[6]  Ioannis G. Kevrekidis,et al.  Model identification of a spatiotemporally varying catalytic reaction , 1993 .

[7]  I.G. Kevrekidis,et al.  Continuous-time nonlinear signal processing: a neural network based approach for gray box identification , 1994, Proceedings of IEEE Workshop on Neural Networks for Signal Processing.

[8]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.

[9]  Carl E. Rasmussen,et al.  In Advances in Neural Information Processing Systems , 2011 .

[10]  Steven A. Orszag,et al.  Scalings in diffusion-driven reactionA+B→C: Numerical simulations by lattice BGK models , 1995 .

[11]  Ioannis G. Kevrekidis,et al.  A comparison of recurrent training algorithms for time series analysis and system identification , 1996 .

[12]  Martin T. Hagan,et al.  Gauss-Newton approximation to Bayesian learning , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[13]  Shiyi Chen,et al.  LATTICE BOLTZMANN METHOD FOR FLUID FLOWS , 2001 .

[14]  Ioannis G. Kevrekidis,et al.  Identification of distributed parameter systems: A neural net based approach , 1998 .

[15]  I. Kevrekidis,et al.  "Coarse" stability and bifurcation analysis using time-steppers: a reaction-diffusion example. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[16]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[17]  C. W. Gear,et al.  Equation-Free, Coarse-Grained Multiscale Computation: Enabling Mocroscopic Simulators to Perform System-Level Analysis , 2003 .

[18]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[19]  J. Boon The Lattice Boltzmann Equation for Fluid Dynamics and Beyond , 2003 .

[20]  Yuan Qi,et al.  Predictive automatic relevance determination by expectation propagation , 2004, ICML.

[21]  T.,et al.  Training Feedforward Networks with the Marquardt Algorithm , 2004 .

[22]  Ann B. Lee,et al.  Geometric diffusions as a tool for harmonic analysis and structure definition of data: diffusion maps. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[23]  Ioannis G. Kevrekidis,et al.  Coarse-grained numerical bifurcation analysis of lattice Boltzmann models , 2005 .

[24]  B. Nadler,et al.  Diffusion maps, spectral clustering and reaction coordinates of dynamical systems , 2005, math/0503445.

[25]  I. Kevrekidis,et al.  Equation-free/Galerkin-free POD-assisted computation of incompressible flows , 2005 .

[26]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[27]  David P. Wipf,et al.  A New View of Automatic Relevance Determination , 2007, NIPS.

[28]  B. Nadler,et al.  Diffusion Maps - a Probabilistic Interpretation for Spectral Embedding and Clustering Algorithms , 2008 .

[29]  Giovanni Samaey,et al.  Equation-free multiscale computation: algorithms and applications. , 2009, Annual review of physical chemistry.

[30]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[31]  Constantinos I. Siettos,et al.  An equation-free approach to agent-based computation: Bifurcation analysis and control of stationary states , 2012 .

[32]  W G Noid,et al.  Perspective: Coarse-grained models for biomolecular systems. , 2013, The Journal of chemical physics.

[33]  Steven L. Brunton,et al.  Data-driven discovery of partial differential equations , 2016, Science Advances.

[34]  Seungjoon Lee,et al.  A resilient and efficient CFD framework: Statistical learning tools for multi-fidelity and heterogeneous information fusion , 2017, J. Comput. Phys..

[35]  Paris Perdikaris,et al.  Machine learning of linear differential equations using Gaussian processes , 2017, J. Comput. Phys..

[36]  Marina Meila,et al.  A regression approach for explaining manifold embedding coordinates , 2018, ArXiv.

[37]  George E. Karniadakis,et al.  Hidden physics models: Machine learning of nonlinear partial differential equations , 2017, J. Comput. Phys..

[38]  Ioannis G. Kevrekidis,et al.  Manifold learning for parameter reduction , 2018, J. Comput. Phys..

[39]  Stephan Hoyer,et al.  Learning data-driven discretizations for partial differential equations , 2018, Proceedings of the National Academy of Sciences.

[40]  Seungjoon Lee,et al.  Linking Gaussian process regression with data-driven manifold embeddings for nonlinear data fusion , 2018, Interface Focus.