Physics-informed machine learning

| Despite great progress in simulating multiphysics problems using the numerical discretization of partial differential equations (PDEs), one still cannot seamlessly incorporate noisy data into existing algorithms, mesh generation remains complex, and highdimensional problems governed by parameterized PDEs cannot be tackled. Moreover, solving inverse problems with hidden physics is often prohibitively expensive and requires different formulations and elaborate computer codes. Machine learning has emerged as a promising alternative, but training deep neural networks requires big data, not always available for scientific problems. Instead, such networks can be trained from additional information obtained by enforcing the physical laws (for example, at random points in the continuous spacetime domain). Such physicsinformed learning integrates (noisy) data and mathematical models, and implements them through neural networks or other kernelbased regression networks. Moreover, it may be possible to design specialized network architectures that automatically satisfy some of the physical invariants for better accuracy, faster training and improved generalization. Here, we review some of the prevailing trends in embedding physics into machine learning, present some of the current capabilities and limitations and discuss diverse applications of physicsinformed learning both for forward and inverse problems, including discovering hidden physics and tackling highdimensional problems. 1Division of Applied

[1]  A. F. Adams,et al.  The Survey , 2021, Dyslexia in Higher Education.

[2]  R. C. Macridis A review , 1963 .

[3]  G. Wahba,et al.  A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines , 1970 .

[4]  F. M. Larkin Gaussian measure in Hilbert space and applications in numerical analysis , 1972 .

[5]  T. J. Rivlin,et al.  Optimal Estimation in Approximation Theory , 1977 .

[6]  Neil F. Dinn Network architectures , 1991, Future Gener. Comput. Syst..

[7]  Ioannis G. Kevrekidis,et al.  DISCRETE- vs. CONTINUOUS-TIME NONLINEAR SIGNAL PROCESSING OF Cu ELECTRODISSOLUTION DATA , 1992 .

[8]  Ronald L. Rivest,et al.  Training a 3-node neural network is NP-complete , 1988, COLT '88.

[9]  Steve Plimpton,et al.  Fast parallel algorithms for short-range molecular dynamics , 1993 .

[10]  I.G. Kevrekidis,et al.  Continuous-time nonlinear signal processing: a neural network based approach for gray box identification , 1994, Proceedings of IEEE Workshop on Neural Networks for Signal Processing.

[11]  Jorge Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[12]  Neural Networks for Chemical Engineers Edited by A. B. Bulsari (Lappeenranta University of Technology, Finland). Elsevier: Amsterdam. 1995. ix + 680 pp. $285.25. ISBN 0-444-820970-3. , 1996 .

[13]  William A. Pearlman,et al.  A SURVEY OF THE STATEOF-THEART AND UTILIZATION OF EMBEDDED, TREEBASED CODING , 1998 .

[14]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[15]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[16]  Ioannis G. Kevrekidis,et al.  Identification of distributed parameter systems: A neural net based approach , 1998 .

[17]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[18]  Klaus Ritter,et al.  Bayesian numerical analysis , 2000 .

[19]  R L Bratman Edge of chaos. , 2002, Journal of the Royal Society of Medicine.

[20]  Dongbin Xiu,et al.  The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations , 2002, SIAM J. Sci. Comput..

[21]  Kirk Martinez,et al.  Environmental Sensor Networks: A revolution in the earth system science? , 2006 .

[22]  Aleksandar Jemcov,et al.  OpenFOAM: A C++ Library for Complex Physics Simulations , 2007 .

[23]  Hans Burkhardt,et al.  Learning Equivariant Functions with Matrix Valued Kernels , 2007, J. Mach. Learn. Res..

[24]  Habib N. Najm,et al.  Stochastic spectral methods for efficient Bayesian solution of inverse problems , 2005, J. Comput. Phys..

[25]  H. Owhadi,et al.  Metric‐based upscaling , 2007 .

[26]  Hod Lipson,et al.  Distilling Free-Form Natural Laws from Experimental Data , 2009, Science.

[27]  Carlo Rovelli,et al.  “Forget time” , 2009, 0903.3832.

[28]  Kevin Stanley McFall,et al.  Artificial Neural Network Method for Solution of Boundary Value Problems With Exact Satisfaction of Arbitrary Boundary Conditions , 2009, IEEE Transactions on Neural Networks.

[29]  Andrew M. Stuart,et al.  Inverse problems: A Bayesian perspective , 2010, Acta Numerica.

[30]  Alán Aspuru-Guzik,et al.  The Harvard Clean Energy Project: Large-Scale Computational Screening and Design of Organic Photovoltaics on the World Community Grid , 2011 .

[31]  Stéphane Mallat,et al.  Invariant Scattering Convolution Networks , 2012, IEEE transactions on pattern analysis and machine intelligence.

[32]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[33]  David J. Schwab,et al.  An exact mapping between the Variational Renormalization Group and Deep Learning , 2014, ArXiv.

[34]  Houman Owhadi,et al.  Bayesian Numerical Homogenization , 2014, Multiscale Model. Simul..

[35]  Anders Logg,et al.  The FEniCS Project Version 1.5 , 2015 .

[36]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[37]  Yann LeCun,et al.  The Loss Surfaces of Multilayer Networks , 2014, AISTATS.

[38]  Stéphane Mallat,et al.  Understanding deep convolutional networks , 2016, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[39]  J. Templeton,et al.  Reynolds averaged turbulence modelling using deep neural networks with embedded invariance , 2016, Journal of Fluid Mechanics.

[40]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Andrew Gordon Wilson,et al.  Deep Kernel Learning , 2015, AISTATS.

[42]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[43]  Stéphane Canu,et al.  Operator-valued Kernels for Learning from Functional Response Data , 2015, J. Mach. Learn. Res..

[44]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[45]  Surya Ganguli,et al.  Exponential expressivity in deep neural networks through transient chaos , 2016, NIPS.

[46]  David J. Schwab,et al.  Supervised Learning with Tensor Networks , 2016, NIPS.

[47]  Michael I. Jordan,et al.  Gradient Descent Only Converges to Minimizers , 2016, COLT.

[48]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[49]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[50]  Lantao Yu,et al.  SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient , 2016, AAAI.

[51]  Qing Nie,et al.  DifferentialEquations.jl – A Performant and Feature-Rich Ecosystem for Solving Differential Equations in Julia , 2017, Journal of Open Research Software.

[52]  Houman Owhadi,et al.  Multigrid with Rough Coefficients and Multiresolution Operator Decomposition from Hierarchical Information Games , 2015, SIAM Rev..

[53]  Samuel S. Schoenholz,et al.  Mean Field Residual Networks: On the Edge of Chaos , 2017, NIPS.

[54]  Paris Perdikaris,et al.  Inferring solutions of differential equations using noisy multi-fidelity data , 2016, J. Comput. Phys..

[55]  E Weinan,et al.  A Proposal on Machine Learning via Dynamical Systems , 2017, Communications in Mathematics and Statistics.

[56]  Paris Perdikaris,et al.  Machine learning of linear differential equations using Gaussian processes , 2017, J. Comput. Phys..

[57]  Lorenzo Rosasco,et al.  Why and when can deep-but not shallow-networks avoid the curse of dimensionality: A review , 2016, International Journal of Automation and Computing.

[58]  Arnulf Jentzen,et al.  A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations , 2018, Memoirs of the American Mathematical Society.

[59]  E Weinan,et al.  Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanics , 2017, Physical review letters.

[60]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[61]  Risi Kondor,et al.  Covariant Compositional Networks For Learning Graphs , 2018, ICLR.

[62]  Steven L. Brunton,et al.  Deep learning for universal linear embeddings of nonlinear dynamics , 2017, Nature Communications.

[63]  Bin Dong,et al.  PDE-Net: Learning PDEs from Data , 2017, ICML.

[64]  Andrew Gordon Wilson,et al.  GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration , 2018, NeurIPS.

[65]  D. S. Reddy,et al.  Prediction of vegetation dynamics using NDVI time series data and LSTM , 2018, Modeling Earth Systems and Environment.

[66]  Prabhat,et al.  Exascale Deep Learning for Climate Analytics , 2018, SC18: International Conference for High Performance Computing, Networking, Storage and Analysis.

[67]  Paris Perdikaris,et al.  Numerical Gaussian Processes for Time-Dependent and Nonlinear Partial Differential Equations , 2017, SIAM J. Sci. Comput..

[68]  Justin A. Sirignano,et al.  DGM: A deep learning algorithm for solving partial differential equations , 2017, J. Comput. Phys..

[69]  Ilias Bilionis,et al.  Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification , 2018, J. Comput. Phys..

[70]  Matthew Johnson,et al.  Compiling machine learning programs via high-level tracing , 2018 .

[71]  Max Welling,et al.  Improved Semantic Segmentation for Histopathology using Rotation Equivariant Convolutional Networks , 2018 .

[72]  Risi Kondor,et al.  Predicting molecular properties with covariant compositional networks. , 2018, The Journal of chemical physics.

[73]  Katharina Krischer,et al.  An Emergent Space for Distributed Data With Hidden Internal Order Through Manifold Learning , 2017, IEEE Access.

[74]  Andrea Montanari,et al.  A mean field view of the landscape of two-layer neural networks , 2018, Proceedings of the National Academy of Sciences.

[75]  Eric Darve,et al.  The Neural Network Approach to Inverse Problems in Differential Equations , 2019, 1901.07758.

[76]  Michael W. Mahoney,et al.  Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction , 2019, ArXiv.

[77]  S. De,et al.  Integrating machine learning and multiscale modeling—perspectives, challenges, and opportunities in the biological, biomedical, and behavioral sciences , 2019, npj Digital Medicine.

[78]  Frank Hutter,et al.  Neural Architecture Search: A Survey , 2018, J. Mach. Learn. Res..

[79]  Jeff Donahue,et al.  Large Scale GAN Training for High Fidelity Natural Image Synthesis , 2018, ICLR.

[80]  Wei Cai Multi-scale Deep Neural Networks for Solving High Dimensional PDEs∗ , 2019 .

[81]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[82]  Marios Mattheakis,et al.  Physical Symmetries Embedded in Neural Networks , 2019, ArXiv.

[83]  David Pfau,et al.  Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks , 2019, Physical Review Research.

[84]  D. Fan,et al.  A robotic Intelligent Towing Tank for learning complex fluid-structure dynamics , 2019, Science Robotics.

[85]  Austin R. Benson,et al.  Neural Jump Stochastic Differential Equations , 2019, NeurIPS.

[86]  George Em Karniadakis,et al.  fPINNs: Fractional Physics-Informed Neural Networks , 2018, SIAM J. Sci. Comput..

[87]  George Em Karniadakis,et al.  Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems , 2018, J. Comput. Phys..

[88]  Yoshua Bengio,et al.  On the Spectral Bias of Neural Networks , 2018, ICML.

[89]  Max Welling,et al.  Gauge Equivariant Convolutional Networks and the Icosahedral CNN 1 , 2019 .

[90]  Peter Bailis,et al.  Equivariant Transformer Networks , 2019, ICML.

[91]  Houman Owhadi,et al.  Kernel Flows: from learning kernels from data into the abyss , 2018, J. Comput. Phys..

[92]  Yibo Yang,et al.  Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems , 2019, Computational Mechanics.

[93]  Levent Sagun,et al.  The jamming transition as a paradigm to understand the loss landscape of deep neural networks , 2018, Physical review. E.

[94]  Baskar Ganapathysubramanian,et al.  Encoding Invariances in Deep Generative Models , 2019, ArXiv.

[95]  R. Batra,et al.  Physically informed artificial neural networks for atomistic modeling of materials , 2018, Nature Communications.

[96]  Jinchao Xu,et al.  MgNet: A unified framework of multigrid and convolutional neural network , 2019, Science China Mathematics.

[97]  Liu Yang,et al.  Neural-net-induced Gaussian process regression for function approximation and PDE solution , 2018, J. Comput. Phys..

[98]  P. Perdikaris,et al.  Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks , 2019 .

[99]  Karthik Ramani,et al.  ConvPDE-UQ: Convolutional neural networks with quantified uncertainty for heterogeneous elliptic partial differential equations on varied domains , 2019, J. Comput. Phys..

[100]  C. Scovel,et al.  Statistical Numerical Approximation , 2019, Notices of the American Mathematical Society.

[101]  Timon Rabczuk,et al.  Transfer learning enhanced physics informed neural network for phase-field modeling of fracture , 2019, Theoretical and Applied Fracture Mechanics.

[102]  Eric F Darve,et al.  Adversarial Numerical Analysis for Inverse Problems , 2019, ArXiv.

[103]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[104]  Charbel Farhat,et al.  Predictive Modeling with Learned Constitutive Laws from Indirect Observations , 2019, 1905.12530.

[105]  Transformer Dissection: An Unified Understanding for Transformer’s Attention via the Lens of Kernel , 2019, EMNLP.

[106]  Ronen Basri,et al.  The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies , 2019, NeurIPS.

[107]  Alexander Koryagin,et al.  PyDEns: a Python Framework for Solving Differential Equations with Neural Networks , 2019, ArXiv.

[108]  Paris Perdikaris,et al.  Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data , 2019, J. Comput. Phys..

[109]  Paris Perdikaris,et al.  Adversarial Uncertainty Quantification in Physics-Informed Neural Networks , 2018, J. Comput. Phys..

[110]  Houman Owhadi,et al.  Operator-Adapted Wavelets, Fast Solvers, and Numerical Homogenization , 2019 .

[111]  Jia Zhao,et al.  Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks , 2020, Communications in Computational Physics.

[112]  R. Juanes,et al.  SciANN: A Keras/TensorFlow wrapper for scientific computations and physics-informed deep learning using artificial neural networks , 2020, Computer Methods in Applied Mechanics and Engineering.

[113]  I. Kevrekidis,et al.  Linking Machine Learning with Multiscale Numerics: Data-Driven Discovery of Homogenized Equations , 2020, JOM.

[114]  Levent Sagun,et al.  Scaling description of generalization with number of parameters in deep learning , 2019, Journal of Statistical Mechanics: Theory and Experiment.

[115]  Wei Cai,et al.  A Phase Shift Deep Neural Network for High Frequency Approximation and Wave Problems , 2019, SIAM Journal on Scientific Computing.

[116]  Mathew Halm,et al.  ContactNets: Learning of Discontinuous Contact Dynamics with Smooth, Implicit Representations , 2020, CoRL.

[117]  Houman Owhadi,et al.  Do ideas have shape? Plato's theory of forms as the continuous limit of artificial neural networks , 2020, ArXiv.

[118]  Mengwu Guo,et al.  An energy-based error bound of physics-informed neural network solutions in elasticity , 2020, ArXiv.

[119]  George Em Karniadakis,et al.  Extraction of mechanical properties of materials through deep learning from instrumented indentation , 2020, Proceedings of the National Academy of Sciences.

[120]  E Weinan,et al.  Pushing the Limit of Molecular Dynamics with Ab Initio Accuracy to 100 Million Atoms with Machine Learning , 2020, SC20: International Conference for High Performance Computing, Networking, Storage and Analysis.

[121]  Justin Sirignano,et al.  DPM: A deep learning PDE augmentation method (with application to large-eddy simulation) , 2019, J. Comput. Phys..

[122]  Eric Darve,et al.  Inverse Modeling of Viscoelasticity Materials using Physics Constrained Learning , 2020, ArXiv.

[123]  G. Karniadakis,et al.  SympNets: Intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems , 2020, Neural Networks.

[124]  Nikola B. Kovachki,et al.  Multipole Graph Neural Operator for Parametric Partial Differential Equations , 2020, NeurIPS.

[125]  D. Bowler,et al.  Large scale and linear scaling DFT with the CONQUEST code. , 2020, The Journal of chemical physics.

[126]  Paris Perdikaris,et al.  Physics‐Informed Deep Neural Networks for Learning Parameters and Constitutive Relationships in Subsurface Flow Problems , 2020, Water Resources Research.

[127]  Eric F Darve,et al.  Coupled Time‐Lapse Full‐Waveform Inversion for Subsurface Flow Problems Using Intrusive Automatic Differentiation , 2019, Water Resources Research.

[128]  G. Karniadakis,et al.  On the Convergence of Physics Informed Neural Networks for Linear Second-Order Elliptic and Parabolic Type PDEs , 2020, Communications in Computational Physics.

[129]  George Em Karniadakis,et al.  hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition , 2020, Computer Methods in Applied Mechanics and Engineering.

[130]  R. Molinaro,et al.  Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs , 2020, ArXiv.

[131]  Jinchao Xu,et al.  Relu Deep Neural Networks and Linear Finite Elements , 2018, Journal of Computational Mathematics.

[132]  Wei Cai,et al.  Multi-scale Deep Neural Network (MscaleDNN) Methods for Oscillatory Stokes Flows in Complex Domains , 2020, Communications in Computational Physics.

[133]  G. Karniadakis,et al.  Error estimates of residual minimization using neural networks for linear PDEs , 2020, ArXiv.

[134]  Yibo Yang,et al.  Bayesian differential programming for robust systems identification under uncertainty , 2020, Proceedings of the Royal Society A.

[135]  George Em Karniadakis,et al.  Physics-Informed Neural Network for Ultrasound Nondestructive Quantification of Surface Breaking Cracks , 2020, Journal of Nondestructive Evaluation.

[136]  P. L. Lagari,et al.  Systematic Construction of Neural Forms for Solving Partial Differential Equations Inside Rectangular Domains, Subject to Initial, Boundary and Interface Conditions , 2020, Int. J. Artif. Intell. Tools.

[137]  Ali Ramadhan,et al.  Universal Differential Equations for Scientific Machine Learning , 2020, ArXiv.

[138]  Ziqi Liu,et al.  Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains , 2020, Communications in Computational Physics.

[139]  Nicholas Geneva,et al.  Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks , 2019, J. Comput. Phys..

[140]  Renato Renner,et al.  Discovering physical concepts with neural networks , 2018, Physical review letters.

[141]  George Em Karniadakis,et al.  Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks , 2019, SIAM J. Sci. Comput..

[142]  P. Perdikaris,et al.  On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks , 2020, ArXiv.

[143]  George Em Karniadakis,et al.  Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations , 2020, Science.

[144]  L. Guibas,et al.  A Point-Cloud Deep Learning Framework for Prediction of Fluid Flow Fields on Irregular Geometries , 2020, ArXiv.

[145]  Ioannis G. Kevrekidis,et al.  Learning emergent PDEs in a learned emergent space , 2020, ArXiv.

[146]  Tao Luo,et al.  Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory , 2020, ArXiv.

[147]  Jonathan T. Barron,et al.  Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains , 2020, NeurIPS.

[148]  G. Karniadakis,et al.  Locally adaptive activation functions with slope recovery term for deep and physics-informed neural networks , 2019, ArXiv.

[149]  Jin Woo Jang,et al.  The model reduction of the Vlasov-Poisson-Fokker-Planck system to the Poisson-Nernst-Planck system via the Deep Neural Network Approach , 2020, ArXiv.

[150]  Grant M. Rotskoff,et al.  Learning with rare data: Using active importance sampling to optimize objectives dominated by rare events , 2020, 2008.06334.

[151]  Jaideep Pathak,et al.  Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics , 2019, Neural Networks.

[152]  G. Karniadakis,et al.  Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems , 2020 .

[153]  Pavlos Protopapas,et al.  NeuroDiffEq: A Python package for solving differential equations with neural networks , 2020, J. Open Source Softw..

[154]  Zheng Ma,et al.  Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks , 2019, Communications in Computational Physics.

[155]  俊一 甘利 5分で分かる!? 有名論文ナナメ読み:Jacot, Arthor, Gabriel, Franck and Hongler, Clement : Neural Tangent Kernel : Convergence and Generalization in Neural Networks , 2020 .

[156]  Zhiping Mao,et al.  DeepXDE: A Deep Learning Library for Solving Differential Equations , 2019, AAAI Spring Symposium: MLPS.

[157]  Jaehoon Lee,et al.  Neural Tangents: Fast and Easy Infinite Neural Networks in Python , 2019, ICLR.

[158]  Eric Darve,et al.  Distributed Machine Learning for Computational Engineering using MPI , 2020, ArXiv.

[159]  D. Hatch,et al.  Uncovering edge plasma dynamics via deep learning from partial observations , 2020 .

[160]  Karthik Kashinath,et al.  Enforcing statistical constraints in generative adversarial networks for modeling chaotic dynamical systems , 2019, J. Comput. Phys..

[161]  George Em Karniadakis,et al.  A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems , 2019, J. Comput. Phys..

[162]  Eric Darve,et al.  ADCME: Learning Spatially-varying Physical Fields using Deep Neural Networks , 2020, ArXiv.

[163]  Timothy M. Hospedales,et al.  Meta-Learning in Neural Networks: A Survey , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[164]  Chao Yang,et al.  PFNN: A Penalty-Free Neural Network Method for Solving a Class of Second-Order Boundary-Value Problems on Complex Geometries , 2020, J. Comput. Phys..

[165]  George Em Karniadakis,et al.  Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators , 2019, Nature Machine Intelligence.

[166]  Nicholas H. Nelsen,et al.  The Random Feature Model for Input-Output Maps between Banach Spaces , 2020, SIAM J. Sci. Comput..

[167]  George Em Karniadakis,et al.  Convergence rate of DeepONets for learning operators arising from advection-diffusion equations , 2021, ArXiv.

[168]  Paris Perdikaris,et al.  Deep learning of free boundary and Stefan problems , 2020, J. Comput. Phys..

[169]  Nikola B. Kovachki,et al.  Fourier Neural Operator for Parametric Partial Differential Equations , 2020, ICLR.

[170]  Suchuan Dong,et al.  A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networks , 2020, J. Comput. Phys..

[171]  George Em Karniadakis,et al.  DeepM&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks , 2020, J. Comput. Phys..

[172]  George E. Karniadakis,et al.  DeepM&Mnet for hypersonics: Predicting the coupled flow and finite-rate chemistry behind a normal shock using neural-network approximation of operators , 2020, J. Comput. Phys..

[173]  Kaiyong Zhao,et al.  AutoML: A Survey of the State-of-the-Art , 2019, Knowl. Based Syst..

[174]  Paris Perdikaris,et al.  Learning Unknown Physics of non-Newtonian Fluids , 2020, Physical Review Fluids.

[175]  J. Darbon,et al.  On some neural network architectures that can represent viscosity solutions of certain high dimensional Hamilton-Jacobi partial differential equations , 2020, J. Comput. Phys..

[176]  Eric Darve,et al.  Learning Constitutive Relations using Symmetric Positive Definite Neural Networks , 2020, J. Comput. Phys..

[177]  Ameya D. Jagtap,et al.  Extended Physics-informed Neural Networks (XPINNs): A Generalized Space-Time Domain Decomposition based Deep Learning Framework for Nonlinear Partial Differential Equations , 2020, AAAI Spring Symposium: MLPS.

[178]  Houman Owhadi,et al.  Learning dynamical systems from data: A simple cross-validation perspective, part I: Parametric kernel flows , 2021, Physica D: Nonlinear Phenomena.

[179]  Liu Yang,et al.  B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data , 2020, J. Comput. Phys..

[180]  Oliver Hennigh,et al.  NVIDIA SimNet^{TM}: an AI-accelerated multi-physics simulation framework , 2020, ArXiv.

[181]  G. Karniadakis,et al.  Flow over an espresso cup: inferring 3-D velocity and pressure fields from tomographic background oriented Schlieren via physics-informed neural networks , 2021, Journal of Fluid Mechanics.

[182]  Yuan Cao,et al.  Towards Understanding the Spectral Bias of Deep Learning , 2019, IJCAI.

[183]  X. Jia,et al.  Physics-Guided Machine Learning for Scientific Discovery: An Application in Simulating Lake Temperature Profiles , 2020, Trans. Data Sci..

[184]  Eric F Darve,et al.  A general approach to seismic inversion with automatic differentiation , 2020, Comput. Geosci..

[185]  George Em Karniadakis,et al.  Error estimates for DeepOnets: A deep learning framework in infinite dimensions , 2021, Transactions of Mathematics and Its Applications.

[186]  Eric C. Cyr,et al.  Thermodynamically consistent physics-informed neural networks for hyperbolic systems , 2020, J. Comput. Phys..

[187]  Paris Perdikaris,et al.  When and why PINNs fail to train: A neural tangent kernel perspective , 2020, J. Comput. Phys..

[188]  N. Zabaras,et al.  Transformers for Modeling Physical Systems , 2020, Neural Networks.

[189]  Eric Darve,et al.  Physics Constrained Learning for Data-driven Inverse Modeling from Sparse Observations , 2020, J. Comput. Phys..

[190]  Generative Ensemble Regression: Learning Particle Dynamics from Observations of Ensembles with Physics-informed Deep Generative Models , 2022, SIAM Journal on Scientific Computing.