Transfer learning based multi-fidelity physics informed deep neural network

For many systems in science and engineering, the governing differential equation is either not known or known in an approximate sense. Analyses and design of such systems are governed by data collected from the field and/or laboratory experiments. This challenging scenario is further worsened when data-collection is expensive and time-consuming. To address this issue, this paper presents a novel multi-fidelity physics informed deep neural network (MF-PIDNN). The framework proposed is particularly suitable when the physics of the problem is known in an approximate sense (low-fidelity physics) and only a few high-fidelity data are available. MF-PIDNN blends physics informed and data-driven deep learning techniques by using the concept of transfer learning. The approximate governing equation is first used to train a low-fidelity physics informed deep neural network. This is followed by transfer learning where the low-fidelity model is updated by using the available high-fidelity data. MF-PIDNN is able to encode useful information on the physics of the problem from the {\it approximate} governing differential equation and hence, provides accurate prediction even in zones with no data. Additionally, no low-fidelity data is required for training this model. Applicability and utility of MF-PIDNN are illustrated in solving four benchmark reliability analysis problems. Case studies to illustrate interesting features of the proposed approach are also presented.

[1]  Bruno Sudret,et al.  Global sensitivity analysis using polynomial chaos expansions , 2008, Reliab. Eng. Syst. Saf..

[2]  R. L. Iman,et al.  Latin hypercube sampling (program user's guide). [LHC, in FORTRAN] , 1980 .

[3]  Souvik Chakraborty,et al.  Polynomial Correlated Function Expansion , 2017 .

[4]  Jens Lorenz,et al.  Nonlinear singular perturbation problems and the Engquist-Osher difference scheme , 1981 .

[5]  Subhayan De,et al.  On transfer learning of neural networks using bi-fidelity data for uncertainty propagation , 2020, International Journal for Uncertainty Quantification.

[6]  George Em Karniadakis,et al.  Supersensitivity due to uncertain boundary conditions , 2004 .

[7]  Atin Roy,et al.  Support vector regression based metamodel by sequential adaptive sampling for reliability analysis of structures , 2020, Reliab. Eng. Syst. Saf..

[8]  Nicholas Zabaras,et al.  Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification , 2018, J. Comput. Phys..

[9]  Loic Le Gratiet,et al.  RECURSIVE CO-KRIGING MODEL FOR DESIGN OF COMPUTER EXPERIMENTS WITH MULTIPLE LEVELS OF FIDELITY , 2012, 1210.0686.

[10]  C. Farhat International Journal for Numerical Methods in Engineering , 2019 .

[11]  Benjamin Peherstorfer,et al.  Survey of multifidelity methods in uncertainty propagation, inference, and optimization , 2018, SIAM Rev..

[12]  Rampi Ramprasad,et al.  Multifidelity Information Fusion with Machine Learning: A Case Study of Dopant Formation Energies in Hafnia. , 2019, ACS applied materials & interfaces.

[13]  K. Ramesh,et al.  Digital photoelasticity of glass: A comprehensive review , 2016 .

[14]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[15]  Dongbin Xiu,et al.  A neural network approach for uncertainty quantification for time-dependent problems with random parameters , 2019, ArXiv.

[16]  Andreas C. Damianou,et al.  Nonlinear information fusion algorithms for data-efficient multi-fidelity modelling , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[17]  Geoffrey T. Parks,et al.  Multi-fidelity non-intrusive polynomial chaos based on regression , 2016 .

[18]  Klaus Ritter,et al.  Adaptive Multilevel Monte Carlo Approximation of Distribution Functions , 2017, 1706.06869.

[19]  Reuven Y. Rubinstein,et al.  Simulation and the Monte Carlo Method , 1981 .

[20]  Irfan Kaymaz,et al.  Application Of Kriging Method To Structural Reliability Problems , 2005 .

[21]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[22]  Souvik Chakraborty,et al.  Sequential experimental design based generalised ANOVA , 2016, J. Comput. Phys..

[23]  Nam H. Kim,et al.  Issues in Deciding Whether to Use Multifidelity Surrogates , 2019, AIAA Journal.

[24]  Anders Logg,et al.  The FEniCS Project Version 1.5 , 2015 .

[25]  Achintya Haldar,et al.  Reliability Assessment Using Stochastic Finite Element Analysis , 2000 .

[26]  Michael B. Giles,et al.  Multilevel Monte Carlo Path Simulation , 2008, Oper. Res..

[27]  Dmitry K. Polyushkin,et al.  Ultrafast machine vision with 2D material neural network image sensors , 2020, Nature.

[28]  Sondipon Adhikari,et al.  A Surrogate Based Multi-fidelity Approach for Robust Design Optimization , 2017 .

[29]  Yan Wang,et al.  Multi-Fidelity Physics-Constrained Neural Network and Its Application in Materials Modeling , 2019, Journal of Mechanical Design.

[30]  Ilias Bilionis,et al.  Multi-output separable Gaussian process: Towards an efficient, fully Bayesian paradigm for uncertainty quantification , 2013, J. Comput. Phys..

[31]  Loic Le Gratiet,et al.  Multi-fidelity Gaussian process regression for computer experiments , 2013 .

[32]  Daniele Venturi,et al.  Multifidelity Information Fusion Algorithms for High-Dimensional Systems and Massive Data sets , 2016, SIAM J. Sci. Comput..

[33]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[34]  Martin T. Hagan,et al.  Neural network design , 1995 .

[35]  Wolfgang A. Wall,et al.  A Generalized Probabilistic Learning Approach for Multi-Fidelity Uncertainty Propagation in Complex Physical Simulations , 2020, ArXiv.

[36]  Sondipon Adhikari,et al.  A Critical Assessment of Kriging Model Variants for High-Fidelity Uncertainty Quantification in Dynamics of composite Shells , 2016, Archives of Computational Methods in Engineering.

[37]  Souvik Chakraborty,et al.  An efficient algorithm for building locally refined hp - adaptive H-PCFE: Application to uncertainty quantification , 2017, J. Comput. Phys..

[38]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[39]  Anupam Chakrabarti,et al.  A simple homogenization scheme for 3D finite element analysis of composite bolted joints , 2015 .

[40]  Kenny Chowdhary,et al.  Machine Learning Models of Errors in Large Eddy Simulation Predictions of Surface Pressure Fluctuations , 2017 .

[41]  T. Dhaene,et al.  Variable-Fidelity Electromagnetic Simulations and Co-Kriging for Accurate Modeling of Antennas , 2013, IEEE Transactions on Antennas and Propagation.

[42]  George Em Karniadakis,et al.  A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems , 2019, J. Comput. Phys..

[43]  Dongbin Xiu,et al.  The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations , 2002, SIAM J. Sci. Comput..

[44]  R. M. Natal Jorge,et al.  Material homogenization technique for composites: A meshless formulation , 2018 .

[45]  Stefan Heinrich,et al.  Multilevel Monte Carlo Methods , 2001, LSSC.

[46]  Satish Chandra,et al.  Kriging based saturation flow models for traffic conditions in Indian cities , 2018, Transportation Research Part A: Policy and Practice.

[47]  Timon Rabczuk,et al.  Transfer learning enhanced physics informed neural network for phase-field modeling of fracture , 2019, Theoretical and Applied Fracture Mechanics.

[48]  Raphael T. Haftka,et al.  Remarks on multi-fidelity surrogates , 2016, Structural and Multidisciplinary Optimization.

[49]  Souvik Chakraborty Simulation free reliability analysis: A physics-informed deep learning based approach , 2020 .

[50]  Barak A. Pearlmutter,et al.  Automatic differentiation in machine learning: a survey , 2015, J. Mach. Learn. Res..

[51]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[52]  Jian-Xun Wang,et al.  A bi-fidelity surrogate modeling approach for uncertainty propagation in three-dimensional hemodynamic simulations , 2019, 1908.10197.

[53]  P Perdikaris,et al.  Multi-fidelity modelling via recursive co-kriging and Gaussian–Markov random fields , 2015, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[54]  Appu Haapio,et al.  Environmental effect of structural solutions and building materials to a building , 2008 .

[55]  Souvik Chakraborty,et al.  Graph-Theoretic-Approach-Assisted Gaussian Process for Nonlinear Stochastic Dynamic Analysis under Generalized Loading , 2019 .

[56]  Jing Li,et al.  Evaluation of failure probability via surrogate models , 2010, J. Comput. Phys..

[57]  Alexey Chernov,et al.  Approximation of probability density functions by the Multilevel Monte Carlo Maximum Entropy method , 2016, J. Comput. Phys..

[58]  Achintya Haldar,et al.  Probability, Reliability and Statistical Methods in Engineering Design (Haldar, Mahadevan) , 1999 .

[59]  Qingfu Zhang,et al.  A multi-fidelity surrogate-model-assisted evolutionary algorithm for computationally expensive optimization problems , 2016, J. Comput. Sci..

[60]  G. Karniadakis,et al.  Multi-fidelity modelling of mixed convection based on experimental correlations and numerical simulations , 2016, Journal of Fluid Mechanics.

[61]  Sriram Narasimhan,et al.  A Gaussian process latent force model for joint input-state estimation in linear structural systems , 2019, Mechanical Systems and Signal Processing.

[62]  Yang Zhiyin,et al.  Large-eddy simulation: Past, present and the future , 2015 .

[63]  D. Lauffenburger,et al.  Autocrine loops with positive feedback enable context-dependent cell signaling. , 2002, American journal of physiology. Cell physiology.

[64]  Souvik Chakraborty,et al.  Hybrid Framework for the Estimation of Rare Failure Event Probability , 2017 .

[65]  Liang Yan,et al.  Adaptive multi-fidelity polynomial chaos approach to Bayesian inference in inverse problems , 2018, J. Comput. Phys..

[66]  Ilias Bilionis,et al.  Multi-output local Gaussian process regression: Applications to uncertainty quantification , 2012, J. Comput. Phys..

[67]  Paris Perdikaris,et al.  Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data , 2019, J. Comput. Phys..

[68]  Satish Chandra,et al.  Kriging-Based Approach for Estimation of Vehicular Speed and Passenger Car Units on an Urban Arterial , 2017 .

[69]  Biswarup Bhattacharyya,et al.  A Critical Appraisal of Design of Experiments for Uncertainty Quantification , 2018 .