GINNs: Graph-Informed Neural Networks for Multiscale Physics

We introduce the concept of a Graph-Informed Neural Network (GINN), a hybrid approach combining deep learning with probabilistic graphical models (PGMs) that acts as a surrogate for physics-based representations of multiscale and multiphysics systems. GINNs address the twin challenges of removing intrinsic computational bottlenecks in physics-based models and generating large data sets for estimating probability distributions of quantities of interest (QoIs) with a high degree of confidence. Both the selection of the complex physics learned by the NN and its supervised learning/prediction are informed by the PGM, which includes the formulation of structured priors for tunable control variables (CVs) to account for their mutual correlations and ensure physically sound CV and QoI distributions. GINNs accelerate the prediction of QoIs essential for simulation-based decision-making where generating sufficient sample data using physics-based models alone is often prohibitively expensive. Using a real-world application grounded in supercapacitor-based energy storage, we describe the construction of GINNs from a Bayesian network-embedded homogenized model for supercapacitor dynamics, and demonstrate their ability to produce kernel density estimates of relevant non-Gaussian, skewed QoIs with tight confidence intervals.

[1]  Ivan P. Parkin,et al.  Tuning the interlayer spacing of graphene laminate films for efficient pore utilization towards compact capacitive energy storage , 2020, Nature Energy.

[2]  George Em Karniadakis,et al.  A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems , 2019, J. Comput. Phys..

[3]  Jinchao Feng,et al.  Non-parametric correlative uncertainty quantification and sensitivity analysis: Application to a Langmuir bimolecular adsorption model , 2018 .

[4]  Nicholas J. Gaul,et al.  Role of pseudo-turbulent stresses in shocked particle clouds and construction of surrogate models for closure , 2018 .

[5]  Petr Plechác,et al.  Path-space variational inference for non-equilibrium coarse-grained systems , 2015, J. Comput. Phys..

[6]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[7]  K. Hanamura,et al.  Theoretical analysis of hydrogen oxidation reaction in solid oxide fuel cell anode based on species territory adsorption model , 2015 .

[8]  Daniel M. Tartakovsky,et al.  Design of nanoporous materials with optimal sorption capacity , 2015 .

[9]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[10]  Nicholas Zabaras,et al.  Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification , 2018, J. Comput. Phys..

[11]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[12]  Raimund Rolfes,et al.  Neural network assisted multiscale analysis for the elastic properties prediction of 3D braided composites under uncertainty , 2018 .

[13]  Mark E. Davis,et al.  Fundamentals of Chemical Reaction Engineering , 2002 .

[14]  Dayadeep S. Monder,et al.  First-Principles Study of Oxygen Transfer and Hydrogen Oxidation Processes at the Ni-YSZ-Gas Triple Phase Boundaries in a Solid Oxide Fuel Cell Anode , 2015 .

[15]  Alexandre M. Tartakovsky,et al.  Dispersion controlled by permeable surfaces: surface properties and scaling , 2016, Journal of Fluid Mechanics.

[16]  J. Pearl,et al.  Causal inference in statistics , 2016 .

[17]  Daniel M. Tartakovsky,et al.  Global sensitivity analysis of multiscale properties of porous materials , 2018 .

[18]  Mohammad Motamed A multi-fidelity neural network surrogate sampling method for uncertainty quantification , 2019, ArXiv.

[19]  Bingjun Xu,et al.  Electrochemical energy engineering: a new frontier of chemical engineering innovation. , 2014, Annual review of chemical and biomolecular engineering.

[20]  V. R. Joseph,et al.  An Efficient Surrogate Model for Emulation and Physics Extraction of Large Eddy Simulations , 2016, Journal of the American Statistical Association.

[21]  Jitesh H. Panchal,et al.  Simulator-free solution of high-dimensional stochastic elliptic partial differential equations using deep neural networks , 2019, J. Comput. Phys..

[22]  Nicholas J. Gaul,et al.  Evaluation of kriging based surrogate models constructed from mesoscale computations of shock interaction with particles , 2017, J. Comput. Phys..

[23]  Hyuk Lee,et al.  Neural algorithm for solving differential equations , 1990 .

[24]  Yen-Chi Chen,et al.  A tutorial on kernel density estimation and recent advances , 2017, 1704.03924.

[25]  E. Torre,et al.  A general framework for data-driven uncertainty quantification under complex input dependencies using vine copulas , 2017, Probabilistic Engineering Mechanics.

[26]  Michael S. Eldred,et al.  Second-Order Corrections for Surrogate-Based Optimization with Model Hierarchies , 2004 .

[27]  Daniel M. Tartakovsky,et al.  Estimation of distributions via multilevel Monte Carlo with stratified sampling , 2019, J. Comput. Phys..

[28]  Linda C. van der Gaag,et al.  Probabilistic Graphical Models , 2014, Lecture Notes in Computer Science.

[29]  Kaj Nyström,et al.  A unified deep artificial neural network approach to partial differential equations in complex geometries , 2017, Neurocomputing.

[30]  Lyle H. Ungar,et al.  A hybrid neural network‐first principles approach to process modeling , 1992 .

[31]  H. S. Udaykumar,et al.  Machine-learning based multi-scale model for shock-particle interactions , 2019 .

[32]  Daniel M. Tartakovsky,et al.  Effective Ion Diffusion in Charged Nanoporous Materials , 2017 .

[33]  Liam J. Butler,et al.  The role of statistics in data-centric engineering , 2018 .

[34]  Paris Perdikaris,et al.  Adversarial Uncertainty Quantification in Physics-Informed Neural Networks , 2018, J. Comput. Phys..

[35]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[36]  Luning Sun,et al.  Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data , 2019, Computer Methods in Applied Mechanics and Engineering.

[37]  George Em Karniadakis,et al.  Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems , 2018, J. Comput. Phys..

[38]  Paris Perdikaris,et al.  Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data , 2019, J. Comput. Phys..

[39]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[40]  Nicholas J. Gaul,et al.  Evaluation of multifidelity surrogate modeling techniques to construct closure laws for drag in shock-particle interactions , 2018, J. Comput. Phys..

[41]  H. S. Udaykumar,et al.  Modeling mesoscale energy localization in shocked HMX, part I: machine-learned surrogate models for the effects of loading and void sizes , 2018, Shock Waves.

[42]  F. Alexander,et al.  Physics-based statistical learning approach to mesoscopic model selection. , 2015, Physical review. E, Statistical, nonlinear, and soft matter physics.

[43]  Bryan C. Daniels,et al.  Automated adaptive inference of phenomenological dynamical models , 2015, Nature Communications.

[44]  Ilias Bilionis,et al.  Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification , 2018, J. Comput. Phys..

[45]  H. S. Udaykumar,et al.  Evaluation of convergence behavior of metamodeling techniques for bridging scales in multi-scale multimaterial simulation , 2015, J. Comput. Phys..

[46]  Illtyd Trethowan Causality , 1938 .

[47]  Mark W. Verbrugge,et al.  Microstructural Analysis and Mathematical Modeling of Electric Double-Layer Supercapacitors , 2005 .

[48]  Justin A. Sirignano,et al.  DGM: A deep learning algorithm for solving partial differential equations , 2017, J. Comput. Phys..

[49]  Dimitris G. Papageorgiou,et al.  Neural-network methods for boundary value problems with irregular boundaries , 2000, IEEE Trans. Neural Networks Learn. Syst..

[50]  Trevor Hastie,et al.  Statistical Learning with Sparsity: The Lasso and Generalizations , 2015 .

[51]  Sanjeev Mukerjee,et al.  Electrocatalysis of CO tolerance in hydrogen oxidation reaction in PEM fuel cells , 1999 .

[52]  Dirk P. Kroese,et al.  Kernel density estimation via diffusion , 2010, 1011.2602.

[53]  Costas Tsouris,et al.  Electrosorption of ions from aqueous solutions by nanostructured carbon aerogel. , 2002, Journal of colloid and interface science.

[54]  Daniel M. Tartakovsky,et al.  Causality and Bayesian Network PDEs for multiscale representations of porous media , 2019, J. Comput. Phys..

[55]  Kailiang Wu,et al.  Data-Driven Deep Learning of Partial Differential Equations in Modal Space , 2020, J. Comput. Phys..

[56]  Michael S. Triantafyllou,et al.  Deep learning of vortex-induced vibrations , 2018, Journal of Fluid Mechanics.

[57]  D. Tartakovsky,et al.  Markov chain Monte Carlo with neural network surrogates: application to contaminant source identification , 2020, Stochastic Environmental Research and Risk Assessment.

[58]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[59]  Miss A.O. Penney (b) , 1974, The New Yale Book of Quotations.