Integrating Physics-Based Modeling with Machine Learning: A Survey

In this manuscript, we provide a structured and comprehensive overview of techniques to integrate machine learning with physics-based modeling. First, we provide a summary of application areas for which these approaches have been applied. Then, we describe classes of methodologies used to construct physics-guided machine learning models and hybrid physics-machine learning frameworks from a machine learning standpoint. With this foundation, we then provide a systematic organization of these existing techniques and discuss ideas for future research.

[1]  E Weinan,et al.  Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanics , 2017, Physical review letters.

[2]  Steven L Brunton,et al.  Sparse identification of nonlinear dynamics for rapid model recovery. , 2018, Chaos.

[3]  Arne Elofsson,et al.  Deep transfer learning in the assessment of the quality of protein models , 2018, 1804.06281.

[4]  Steven L. Brunton,et al.  Deep learning for universal linear embeddings of nonlinear dynamics , 2017, Nature Communications.

[5]  Douglas B. Lenat,et al.  The Role of Heuristics in Learning by Discovery: Three Case Studies , 1983 .

[6]  Lexing Ying,et al.  Solving for high-dimensional committor functions using artificial neural networks , 2018, Research in the Mathematical Sciences.

[7]  Anuj Karpatne,et al.  Machine Learning for the Geosciences: Challenges and Opportunities , 2017, IEEE Transactions on Knowledge and Data Engineering.

[8]  Charles Blundell,et al.  Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles , 2016, NIPS.

[9]  Yibo Yang,et al.  Physics-informed deep generative models , 2018, ArXiv.

[10]  Frank Hutter,et al.  Neural Architecture Search: A Survey , 2018, J. Mach. Learn. Res..

[11]  Barbara Solenthaler,et al.  Data-driven fluid simulations using regression forests , 2015, ACM Trans. Graph..

[12]  Klaus-Robert Müller,et al.  SchNet: A continuous-filter convolutional neural network for modeling quantum interactions , 2017, NIPS.

[13]  Pierre Gentine,et al.  Deep learning to represent subgrid processes in climate models , 2018, Proceedings of the National Academy of Sciences.

[14]  Jin-Long Wu,et al.  Enforcing Deterministic Constraints on Generative Adversarial Networks for Emulating Physical Systems , 2019 .

[15]  Furong Gao,et al.  Model Migration with Inclusive Similarity for Development of a New Process Model , 2008 .

[16]  Michael Unser,et al.  Convolutional Neural Networks for Inverse Problems in Imaging: A Review , 2017, IEEE Signal Processing Magazine.

[17]  Karthik Duraisamy,et al.  Physics-Informed Probabilistic Learning of Linear Embeddings of Nonlinear Dynamics with Guaranteed Stability , 2019, SIAM J. Appl. Dyn. Syst..

[18]  Steven L. Brunton,et al.  Multiresolution Convolutional Autoencoders , 2020, J. Comput. Phys..

[19]  J. Hesthaven,et al.  Data-driven reduced order modeling for time-dependent problems , 2019, Computer Methods in Applied Mechanics and Engineering.

[20]  Steven L. Brunton,et al.  Koopman operator theory: Past, present, and future , 2017 .

[21]  Max Tegmark,et al.  AI Feynman: A physics-inspired method for symbolic regression , 2019, Science Advances.

[22]  Mike Christie,et al.  Uncertainty quantification for porous media flows , 2006, J. Comput. Phys..

[23]  E Weinan,et al.  End-to-end Symmetry Preserving Inter-atomic Potential Energy Model for Finite and Extended Systems , 2018, NeurIPS.

[24]  Donald Gerwin,et al.  Information processing, data inferences, and scientific generalization , 1974 .

[25]  Naoya Takeishi,et al.  Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition , 2017, NIPS.

[26]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Gene H. Golub,et al.  Matrix computations , 1983 .

[28]  Prasanna Balaprakash,et al.  Time-series learning of latent-space dynamics for reduced-order model closure , 2019, Physica D: Nonlinear Phenomena.

[29]  Omer San,et al.  An artificial neural network framework for reduced order modeling of transient flows , 2018, Commun. Nonlinear Sci. Numer. Simul..

[30]  Patrick Jöckel,et al.  Earth System Modeling , 2012 .

[31]  Chitta Baral,et al.  Integrating Knowledge and Reasoning in Image Understanding , 2019, IJCAI.

[32]  E Weinan,et al.  Solving many-electron Schrödinger equation using deep neural networks , 2018, J. Comput. Phys..

[33]  Omer San,et al.  Machine learning closures for model order reduction of thermal fluids , 2018, Applied Mathematical Modelling.

[34]  Bing Dong,et al.  A hybrid model approach for forecasting future residential electricity consumption , 2016 .

[35]  Steven K. Kauwe,et al.  Machine Learning Prediction of Heat Capacity for Solid Inorganics , 2018, Integrating Materials and Manufacturing Innovation.

[36]  Hoshin Vijai Gupta,et al.  Debates—the future of hydrological sciences: A (common) path forward? Using models and data to learn: A systems theoretic perspective on the future of hydrological science , 2014 .

[37]  Frank Noé,et al.  Time-lagged autoencoders: Deep learning of slow collective variables for molecular kinetics , 2017, The Journal of chemical physics.

[38]  Ilias Bilionis,et al.  Multi-output local Gaussian process regression: Applications to uncertainty quantification , 2012, J. Comput. Phys..

[39]  Mamikon Gulian,et al.  A Survey of Constrained Gaussian Process Regression: Approaches and Implementation Challenges , 2020, ArXiv.

[40]  Mrinal K. Sen,et al.  Unsupervised physics-based neural networks for seismic migration , 2019 .

[41]  Frank Noé,et al.  Variational Koopman models: Slow collective variables and molecular kinetics from short off-equilibrium simulations. , 2016, The Journal of chemical physics.

[42]  Andrew Stuart,et al.  Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High‐Resolution Simulations , 2017, 1709.00037.

[43]  Karthik Kashinath,et al.  Enforcing statistical constraints in generative adversarial networks for modeling chaotic dynamical systems , 2019, J. Comput. Phys..

[44]  B. Frey,et al.  Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning , 2015, Nature Biotechnology.

[45]  Achuta Kadambi,et al.  Deep Shape from Polarization , 2020, ECCV.

[46]  Saibal Mukhopadhyay,et al.  HybridNet: Integrating Model-based and Data-driven Learning to Predict Evolution of Dynamical Systems , 2018, CoRL.

[47]  George Em Karniadakis,et al.  A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems , 2019, J. Comput. Phys..

[48]  X. Jia,et al.  Predicting lake surface water phosphorus dynamics using process-guided machine learning , 2020, Ecological Modelling.

[49]  Liu Yang,et al.  Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations , 2018, SIAM J. Sci. Comput..

[50]  Joachim Denzler,et al.  Deep learning and process understanding for data-driven Earth system science , 2019, Nature.

[51]  Pat Langley,et al.  Data-Driven Discovery of Physical Laws , 1981, Cogn. Sci..

[52]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[53]  Amit Chakraborty,et al.  Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control , 2020, ICLR.

[54]  Paris Perdikaris,et al.  Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations , 2017, ArXiv.

[55]  Matthias Troyer,et al.  Solving the quantum many-body problem with artificial neural networks , 2016, Science.

[56]  Karen Willcox,et al.  Surrogate and reduced-order modeling: a comparison of approaches for large-scale statistical inverse problems [Chapter 7] , 2010 .

[57]  A. Millis,et al.  Machine learning for many-body physics: The case of the Anderson impurity model , 2014, 1408.1143.

[58]  Wolfgang Polifke,et al.  Physics-Informed Echo State Networks for Chaotic Systems Forecasting , 2019, ICCS.

[59]  Jinlong Wu,et al.  Physics-informed machine learning approach for reconstructing Reynolds stress modeling discrepancies based on DNS data , 2016, 1606.07987.

[60]  Lei Yan,et al.  Physics-based Neural Networks for Shape from Polarization , 2019, ArXiv.

[61]  Michael Unser,et al.  Deep Convolutional Neural Network for Inverse Problems in Imaging , 2016, IEEE Transactions on Image Processing.

[62]  Max Welling,et al.  Spherical CNNs , 2018, ICLR.

[63]  Justin A. Sirignano,et al.  DGM: A deep learning algorithm for solving partial differential equations , 2017, J. Comput. Phys..

[64]  Xiaoou Tang,et al.  Facial Landmark Detection by Deep Multi-task Learning , 2014, ECCV.

[65]  R. Rosenfeld.  Nature , 2009, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[66]  Yibo Yang,et al.  Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems , 2019, Computational Mechanics.

[67]  Zhizhen Zhao,et al.  Spatiotemporal Feature Extraction with Data-Driven Koopman Operators , 2015, FE@NIPS.

[68]  Yi Ren,et al.  Improving direct physical properties prediction of heterogeneous materials from imaging data via convolutional neural network and a morphology-aware generative model , 2017, Computational Materials Science.

[69]  Seiji Tsutsumi,et al.  Feedback Control of Karman Vortex Shedding from a Cylinder using Deep Reinforcement Learning , 2018, 2018 Flow Control Conference.

[70]  Marios Mattheakis,et al.  Physical Symmetries Embedded in Neural Networks , 2019, ArXiv.

[71]  O. Bagasra,et al.  Proceedings of the National Academy of Sciences , 1914, Science.

[72]  Risi Kondor,et al.  Cormorant: Covariant Molecular Neural Networks , 2019, NeurIPS.

[73]  Wei Chen,et al.  BézierGAN: Automatic Generation of Smooth Curves from Interpretable Low-Dimensional Parameters , 2018, ArXiv.

[74]  Patrick Doherty,et al.  Model-Based Reinforcement Learning in Continuous Environments Using Real-Time Constrained Optimization , 2015, AAAI.

[75]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[76]  Manuel Remelhe,et al.  Between the Poles of Data‐Driven and Mechanistic Modeling for Process Operation , 2017 .

[77]  Furong Gao,et al.  Process similarity and developing new process models through migration , 2009 .

[78]  Gianluigi Rozza,et al.  Reduced Order Methods for Modeling and Computational Reduction , 2013 .

[79]  C. Farhat,et al.  Interpolation Method for Adapting Reduced-Order Models and Application to Aeroelasticity , 2008 .

[80]  Andrea Manzoni,et al.  Accurate Solution of Bayesian Inverse Uncertainty Quantification Problems Combining Reduced Basis Methods and Reduction Error Models , 2016, SIAM/ASA J. Uncertain. Quantification.

[81]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[82]  Pascal Van Hentenryck,et al.  Predicting AC Optimal Power Flows: Combining Deep Learning and Lagrangian Dual Methods , 2020, AAAI.

[83]  Michael Zibulevsky,et al.  Self-supervised learning of inverse problem solvers in medical imaging , 2019, DART/MIL3ID@MICCAI.

[84]  Ashish Kapoor,et al.  AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles , 2017, FSR.

[85]  Daniel Walton,et al.  The Atmospheric River Tracking Method Intercomparison Project (ARTMIP): Quantifying Uncertainties in Atmospheric River Climatology , 2019, Journal of Geophysical Research: Atmospheres.

[86]  Vladimir M. Krasnopolsky,et al.  Complex hybrid models combining deterministic and machine learning components for numerical climate modeling and weather prediction , 2006, Neural Networks.

[87]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[88]  Mykel J. Kochenderfer,et al.  Deep Variational Koopman Models: Inferring Koopman Observations for Uncertainty-Aware Dynamics Modeling and Control , 2019, IJCAI.

[89]  Arnulf Jentzen,et al.  Solving high-dimensional partial differential equations using deep learning , 2017, Proceedings of the National Academy of Sciences.

[90]  Linfeng Zhang,et al.  DeePCG: Constructing coarse-grained models via deep neural networks. , 2018, The Journal of chemical physics.

[91]  Steven L. Brunton,et al.  Data-driven discovery of partial differential equations , 2016, Science Advances.

[92]  C. Deser,et al.  Variability , 2020, Encyclopedia of Continuum Mechanics.

[93]  Petros Koumoutsakos,et al.  Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks , 2018, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[94]  Roberto Paolucci,et al.  Broadband Ground Motions from 3D Physics-Based Numerical Simulations Using Artificial Neural Networks , 2018 .

[95]  Rose Yu,et al.  Incorporating Symmetry into Deep Dynamics Models for Improved Generalization , 2020, ArXiv.

[96]  Gustau Camps-Valls,et al.  Latent force models for earth observation time series prediction , 2016, 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP).

[97]  Nikhil Muralidhar,et al.  PhyNet: Physics Guided Neural Networks for Particle Drag Force Prediction in Assembly , 2020, SDM.

[98]  Andrew Jaegle,et al.  Hamiltonian Generative Networks , 2020, ICLR.

[99]  Luca Martino,et al.  Joint Gaussian Processes for Biophysical Parameter Retrieval , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[100]  Razvan Pascanu,et al.  Interaction Networks for Learning about Objects, Relations and Physics , 2016, NIPS.

[101]  Jinkyoo Park,et al.  Physics-induced graph neural network: An application to wind-farm power estimation , 2019, Energy.

[102]  Clarence W. Rowley,et al.  Linearly-Recurrent Autoencoder Networks for Learning Dynamics , 2017, SIAM J. Appl. Dyn. Syst..

[103]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[104]  Karthik Duraisamy,et al.  A paradigm for data-driven predictive modeling using field inversion and machine learning , 2016, J. Comput. Phys..

[105]  R. Hogan,et al.  Journal of Advances in Modeling Earth Systems A Flexible and Efficient Radiation Scheme for the ECMWFModel , 2018 .

[106]  Li Li,et al.  Tensor Field Networks: Rotation- and Translation-Equivariant Neural Networks for 3D Point Clouds , 2018, ArXiv.

[107]  G. R. Schleder,et al.  From DFT to machine learning: recent approaches to materials science–a review , 2019, Journal of Physics: Materials.

[108]  Tapan Mukerji,et al.  Prestack and poststack inversion using a physics-guided convolutional neural network , 2019, Interpretation.

[109]  S. E. Haupt,et al.  Using Artificial Intelligence to Improve Real-Time Decision-Making for High-Impact Weather , 2017 .

[110]  Andrew J. Connolly,et al.  Statistics, Data Mining, and Machine Learning in Astronomy , 2014 .

[111]  J N Kutz,et al.  Model selection for dynamical systems via sparse regression and information criteria , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[112]  Neil D. Lawrence,et al.  Latent Force Models , 2009, AISTATS.

[113]  Jun Zhang,et al.  Data-driven discovery of governing equations for fluid dynamics based on molecular simulation , 2020, Journal of Fluid Mechanics.

[114]  J. Wiens,et al.  A Domain Guided CNN Architecture for Predicting Age from Structural Brain Images , 2018, MLHC.

[115]  Sergey Levine,et al.  Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[116]  William L. Jorgensen,et al.  Journal of Chemical Information and Modeling , 2005, J. Chem. Inf. Model..

[117]  Ioannis G Kevrekidis,et al.  Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator. , 2017, Chaos.

[118]  Haoji Hu,et al.  Physics-Guided Deep Neural Networks for Power Flow Analysis , 2020, IEEE Transactions on Power Systems.

[119]  Mark A. Kramer,et al.  Modeling chemical processes using prior knowledge and neural networks , 1994 .

[120]  J. Nathan Kutz,et al.  Deep learning in fluid dynamics , 2017, Journal of Fluid Mechanics.

[121]  Scott T. Miller,et al.  Physics-enhanced neural networks learn order and chaos. , 2019, Physical review. E.

[122]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[123]  Lars Kotthoff,et al.  Automated Machine Learning: Methods, Systems, Challenges , 2019, The Springer Series on Challenges in Machine Learning.

[124]  J. Behler.  Neural network potential-energy surfaces in chemistry: a tool for large-scale simulations. , 2011, Physical chemistry chemical physics : PCCP.

[125]  A. Gross,et al.  Representing high-dimensional potential-energy surfaces for reactions at surfaces by neural networks , 2004 .

[126]  Hod Lipson,et al.  Automated reverse engineering of nonlinear dynamical systems , 2007, Proceedings of the National Academy of Sciences.

[127]  Eric Horvitz,et al.  A Deep Hybrid Model for Weather Forecasting , 2015, KDD.

[128]  Jun Zhang,et al.  Accelerating Physics-Informed Neural Network Training with Prior Dictionaries , 2020, ArXiv.

[129]  Balaji Srinivasan,et al.  Solution of Biharmonic Equation in Complicated Geometries With Physics Informed Extreme Learning Machine , 2020, J. Comput. Inf. Sci. Eng..

[130]  Faming Liang,et al.  Estimating uncertainty of streamflow simulation using Bayesian neural networks , 2009 .

[131]  Nils Thuerey,et al.  tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow , 2018 .

[132]  Furong Gao,et al.  Bayesian improved model migration methodology for fast process modeling by incorporating prior information , 2015 .

[133]  Julia Ling,et al.  Physics-Informed Machine Learning for Predictive Turbulence Modeling: A Priori Assessment of Prediction Confidence , 2016 .

[134]  A. Pukrittayakamee,et al.  Simultaneous fitting of a potential-energy surface and its corresponding force fields using feedforward neural networks. , 2009, The Journal of chemical physics.

[135]  Peter Sollich,et al.  Accurate interatomic force fields via machine learning with covariant kernels , 2016, 1611.03877.

[136]  V. Prybutok,et al.  A neural network model forecasting for prediction of daily maximum ozone concentration in an industrialized urban area. , 1996, Environmental pollution.

[137]  Amir Barati Farimani,et al.  Weakly-Supervised Deep Learning of Heat Transport via Physics Informed Loss , 2018, ArXiv.

[138]  Eldad Haber,et al.  Deep Neural Networks Motivated by Partial Differential Equations , 2018, Journal of Mathematical Imaging and Vision.

[139]  Furong Gao,et al.  Bayesian migration of Gaussian process regression for rapid process modeling and optimization , 2011 .

[140]  Heng Xiao,et al.  A Priori Assessment of Prediction Confidence for Data-Driven Turbulence Modeling , 2017, Flow, Turbulence and Combustion.

[141]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[142]  P. Gentine,et al.  Evaluation and machine learning improvement of global hydrological model-based flood simulations , 2019, Environmental Research Letters.

[143]  Giulia Marcucci,et al.  Machine learning inverse problem for topological photonics , 2018, Communications Physics.

[144]  Petros Koumoutsakos,et al.  Data-assisted reduced-order modeling of extreme events in complex dynamical systems , 2018, PloS one.

[145]  Evan B. Goldstein,et al.  Data-driven components in a model of inner-shelf sorted bedforms: a new hybrid model , 2013 .

[146]  George Em Karniadakis,et al.  Systems biology informed deep learning for inferring parameters and hidden dynamics , 2019, bioRxiv.

[147]  Ghassan AlRegib,et al.  Successful leveraging of image processing and machine learning in seismic structural interpretation: A review , 2018, The Leading Edge.

[148]  Steven L. Brunton,et al.  Constrained sparse Galerkin regression , 2016, Journal of Fluid Mechanics.

[149]  Lexing Ying,et al.  SwitchNet: a neural network model for forward and inverse scattering problems , 2018, SIAM J. Sci. Comput..

[150]  L. Magri,et al.  First-principles machine learning modelling of COVID-19 , 2020, 2004.09478.

[151]  Nima Tajbakhsh,et al.  Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? , 2016, IEEE Transactions on Medical Imaging.

[152]  Dan Givoli,et al.  Obstacle segmentation based on the wave equation and deep learning , 2020, J. Comput. Phys..

[153]  G. Kerschen,et al.  The Method of Proper Orthogonal Decomposition for Dynamical Characterization and Order Reduction of Mechanical Systems: An Overview , 2005 .

[154]  Kevin B. Flores,et al.  Hybrid modeling and prediction of dynamical systems , 2017, PLoS Comput. Biol..

[155]  Mohammad Rajabi,et al.  Uncertainty-based simulation-optimization using Gaussian process emulation: Application to coastal groundwater management , 2017 .

[156]  Anuj Karpatne,et al.  Process‐Guided Deep Learning Predictions of Lake Water Temperature , 2019, Water Resources Research.

[157]  Paris Perdikaris,et al.  Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations , 2017, ArXiv.

[158]  Paul Raccuglia,et al.  Machine-learning-assisted materials discovery using failed experiments , 2016, Nature.

[159]  Jason Yosinski,et al.  Hamiltonian Neural Networks , 2019, NeurIPS.

[160]  Hao Wu,et al.  VAMPnets for deep learning of molecular kinetics , 2017, Nature Communications.

[161]  C. Folberth,et al.  Spatio-temporal downscaling of gridded crop model yield estimates based on machine learning , 2019, Agricultural and Forest Meteorology.

[162]  Noah D. Brenowitz,et al.  Spatially Extended Tests of a Neural Network Parametrization Trained by Coarse‐Graining , 2019, Journal of Advances in Modeling Earth Systems.

[163]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[164]  Michael W. Mahoney,et al.  Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction , 2019, ArXiv.

[165]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[166]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[167]  Ahmed H. Elsheikh,et al.  DR-RNN: A deep residual recurrent neural network for model reduction , 2017, ArXiv.

[168]  T. Liao,et al.  Metaheuristic-based inverse design of materials – A survey , 2020 .

[169]  Alaeddin Malek,et al.  Numerical solution for high order differential equations using a hybrid neural network - Optimization method , 2006, Appl. Math. Comput..

[170]  Luke de Oliveira,et al.  Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis , 2017, Computing and Software for Big Science.

[171]  Li Zheng,et al.  Inverse-designed spinodoid metamaterials , 2020, npj Computational Materials.

[172]  Weile Jia,et al.  Deep Density: circumventing the Kohn-Sham equations via symmetry preserving neural networks , 2019, J. Comput. Phys..

[173]  Yuankai Wu,et al.  Inductive Graph Neural Networks for Spatiotemporal Kriging , 2020, AAAI.

[174]  Anuj Karpatne,et al.  Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling , 2017, ArXiv.

[175]  Demetris Stathakis,et al.  Neural networks as a tool for constructing continuous NDVI time series from AVHRR and MODIS , 2008 .

[176]  Michael S. Triantafyllou,et al.  Deep learning of vortex-induced vibrations , 2018, Journal of Fluid Mechanics.

[177]  Wojciech Zaremba,et al.  Domain randomization for transferring deep neural networks from simulation to the real world , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[178]  Lijing Wang,et al.  TDEFSI: Theory Guided Deep Learning Based Epidemic Forecasting with Synthetic Information , 2020, ACM Trans. Spatial Algorithms Syst..

[179]  Dawei Han,et al.  Machine Learning Techniques for Downscaling SMOS Satellite Soil Moisture Using MODIS Land Surface Temperature for Hydrological Application , 2013, Water Resources Management.

[180]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[181]  B. Saghafian,et al.  Downscaling Satellite Precipitation Estimates With Multiple Linear Regression, Artificial Neural Networks, and Spline Interpolation Techniques , 2019, Journal of Geophysical Research: Atmospheres.

[182]  Nils Thürey,et al.  tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow , 2018, ACM Trans. Graph..

[183]  Klaus-Robert Müller,et al.  Assessment and Validation of Machine Learning Methods for Predicting Molecular Atomization Energies. , 2013, Journal of chemical theory and computation.

[184]  Prabhat,et al.  Highly-Ccalable, Physics-Informed GANs for Learning Solutions of Stochastic PDEs , 2019, 2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS).

[185]  R. Biswas,et al.  A framework for glass-box physics rule learner and its application to nano-scale phenomena , 2020 .

[186]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[187]  Zhihai He,et al.  A Comprehensive Survey on Geometric Deep Learning , 2020, IEEE Access.

[188]  Wei Li,et al.  Convolutional Neural Networks for Steady Flow Approximation , 2016, KDD.

[189]  Yan Wang,et al.  Multi-Fidelity Physics-Constrained Neural Network and Its Application in Materials Modeling , 2019, Journal of Mechanical Design.

[190]  Noah D. Brenowitz,et al.  Prognostic Validation of a Neural Network Unified Physics Parameterization , 2018, Geophysical Research Letters.

[191]  Rahul Rai,et al.  Driven by Data or Derived Through Physics? A Review of Hybrid Physics Guided Machine Learning Techniques With Cyber-Physical System (CPS) Focus , 2020, IEEE Access.

[192]  Baskar Ganapathysubramanian,et al.  Encoding Invariances in Deep Generative Models , 2019, ArXiv.

[193]  Karthik Ramani,et al.  ConvPDE-UQ: Convolutional neural networks with quantified uncertainty for heterogeneous elliptic partial differential equations on varied domains , 2019, J. Comput. Phys..

[194]  Kookjin Lee,et al.  Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders , 2018, J. Comput. Phys..

[195]  Nicholas Zabaras,et al.  Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification , 2018, J. Comput. Phys..

[196]  E Kaiser,et al.  Sparse identification of nonlinear dynamics for model predictive control in the low-data limit , 2017, Proceedings of the Royal Society A.

[197]  P. Nowack,et al.  Using machine learning to build temperature-based ozone parameterizations for climate sensitivity simulations , 2018, Environmental Research Letters.

[198]  J. J. Soares Neto,et al.  The fitting of potential energy surfaces using neural networks. Application to the study of the photodissociation processes , 1998 .

[199]  Jonathan R. Holland,et al.  Field Inversion and Machine Learning With Embedded Neural Networks: Physics-Consistent Neural Network Training , 2019, AIAA Aviation 2019 Forum.

[200]  Yong Yu,et al.  Long Text Generation via Adversarial Training with Leaked Information , 2017, AAAI.

[201]  Sangram Ganguly,et al.  DeepSD: Generating High Resolution Climate Change Projections through Single Image Super-Resolution , 2017, KDD.

[202]  Michael T. Manry,et al.  Inversion of Surface Parameters Using Fast Learning Neural Networks , 1992, [Proceedings] IGARSS '92 International Geoscience and Remote Sensing Symposium.

[203]  Feng Lin,et al.  Low-Dose CT With a Residual Encoder-Decoder Convolutional Neural Network , 2017, IEEE Transactions on Medical Imaging.

[204]  Stephan Hoyer,et al.  Learning data-driven discretizations for partial differential equations , 2018, Proceedings of the National Academy of Sciences.

[205]  Jeong-Hwan Kim,et al.  Deep learning for multi-year ENSO forecasts , 2019, Nature.

[206]  N. Phan-Thien,et al.  Neural-network-based approximations for solving partial differential equations , 1994 .

[207]  H. Simon,et al.  Rediscovering Chemistry with the Bacon System , 1983 .

[208]  Hua Bao,et al.  Predicting the effective thermal conductivities of composite materials and porous media by machine learning methods , 2018, International Journal of Heat and Mass Transfer.

[209]  George E. Karniadakis,et al.  Hidden physics models: Machine learning of nonlinear partial differential equations , 2017, J. Comput. Phys..

[210]  Xinlei Chen,et al.  PGA: Physics Guided and Adaptive Approach for Mobile Fine-Grained Air Pollution Estimation , 2018, UbiComp/ISWC Adjunct.

[211]  Jitesh H. Panchal,et al.  Simulator-free solution of high-dimensional stochastic elliptic partial differential equations using deep neural networks , 2019, J. Comput. Phys..

[212]  Liu Yang,et al.  B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data , 2020, J. Comput. Phys..

[213]  Nathan DeBardeleben,et al.  Physics-Informed Machine Learning for DRAM Error Modeling , 2018, 2018 IEEE International Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems (DFT).

[214]  Achuta Kadambi,et al.  Blending Diverse Physical Priors with Neural Networks , 2019, ArXiv.

[215]  Eugenia Kalnay,et al.  Atmospheric Modeling, Data Assimilation and Predictability , 2002 .

[216]  Vivek Dua,et al.  An Artificial Neural Network approximation based decomposition approach for parameter estimation of system of ordinary differential equations , 2011, Comput. Chem. Eng..

[217]  U. Schoepf,et al.  Coronary CT Angiography-derived Fractional Flow Reserve: Machine Learning Algorithm versus Computational Fluid Dynamics Modeling. , 2018, Radiology.

[218]  Heiga Zen,et al.  WaveNet: A Generative Model for Raw Audio , 2016, SSW.

[219]  Steven L. Brunton,et al.  Deep learning of dynamics and signal-noise decomposition with time-stepping constraints , 2018, J. Comput. Phys..

[220]  George Em Karniadakis,et al.  PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs , 2019, ArXiv.

[221]  Neil D. Lawrence,et al.  Modelling transcriptional regulation using Gaussian Processes , 2006, NIPS.

[222]  Hod Lipson,et al.  Distilling Free-Form Natural Laws from Experimental Data , 2009, Science.

[223]  Paris Perdikaris,et al.  Adversarial Uncertainty Quantification in Physics-Informed Neural Networks , 2018, J. Comput. Phys..

[224]  Markus H. Gross,et al.  Deep Fluids: A Generative Network for Parameterized Fluid Simulations , 2018, Comput. Graph. Forum.

[225]  Gang Wang,et al.  REAL-TIME POWER SYSTEM STATE ESTIMATION VIA DEEP UNROLLED NEURAL NETWORKS , 2018, 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[226]  Nagiza F. Samatova,et al.  Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data , 2016, IEEE Transactions on Knowledge and Data Engineering.

[227]  Igor Mezi'c,et al.  Correspondence between Koopman mode decomposition, resolvent mode decomposition, and invariant solutions of the Navier-Stokes equations , 2016, 1603.08378.

[228]  Patrick Gallinari,et al.  Deep learning for physical processes: incorporating prior scientific knowledge , 2017, ICLR.

[229]  Nicholas Geneva,et al.  Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks , 2019, J. Comput. Phys..

[230]  Nikhil Muralidhar,et al.  Physics-guided Design and Learning of Neural Networks for Predicting Drag Force on Particle Suspensions in Moving Fluids , 2019, ArXiv.

[231]  Amir Barati Farimani,et al.  Deep Learning the Physics of Transport Phenomena , 2017, ArXiv.

[232]  P. O'Gorman,et al.  Using Machine Learning to Parameterize Moist Convection: Potential for Modeling of Climate, Climate Change, and Extreme Events , 2018, Journal of Advances in Modeling Earth Systems.

[233]  Joshua B. Tenenbaum,et al.  A Compositional Object-Based Approach to Learning Physical Dynamics , 2016, ICLR.

[234]  Dan Klein,et al.  Fast Exact Inference with a Factored Model for Natural Language Parsing , 2002, NIPS.

[235]  Anuj Karpatne,et al.  Physics-Guided Architecture (PGA) of Neural Networks for Quantifying Uncertainty in Lake Temperature Modeling , 2019, SDM.

[236]  Omer San,et al.  Neural network closures for nonlinear model order reduction , 2017, Adv. Comput. Math..

[237]  Qiang Zheng,et al.  Physics-informed semantic inpainting: Application to geostatistical modeling , 2020, J. Comput. Phys..

[238]  Yang Liu,et al.  Physics-guided Convolutional Neural Network (PhyCNN) for Data-driven Seismic Response Modeling , 2019, Engineering Structures.

[239]  David Pfau,et al.  Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks , 2019, Physical Review Research.

[240]  William W. Hsieh,et al.  Machine Learning Methods in the Environmental Sciences: Neural Networks and Kernels , 2009 .

[241]  Ivan Bratko,et al.  Equation discovery with ecological applications , 1999 .

[242]  Neil D. Lawrence,et al.  Gaussian process modelling of latent chemical species: applications to inferring transcription factor activities , 2008, ECCB.

[243]  Maarten V. de Hoop,et al.  Machine learning for data-driven discovery in solid Earth geoscience , 2019, Science.

[244]  Marco Davidovic,et al.  Using Physics-Informed Super-Resolution Generative Adversarial Networks for Subgrid Modeling in Turbulent Reactive Flows , 2019, ArXiv.

[245]  Gianluigi Rozza,et al.  Model Order Reduction in Fluid Dynamics: Challenges and Perspectives , 2014 .

[246]  I. Mezić.  Spectral Properties of Dynamical Systems, Model Reduction and Decompositions , 2005 .

[247]  Hyuk Lee,et al.  Neural algorithm for solving differential equations , 1990 .

[248]  Urban Forssell,et al.  Combining Semi-Physical and Neural Network Modeling: An Example ofIts Usefulness , 1997 .

[249]  Linfeng Liu,et al.  Kriging Convolutional Networks , 2020, AAAI.

[250]  Ahmed H. Elsheikh,et al.  Parametrization and generation of geological models with generative adversarial networks , 2017, 1708.01810.

[251]  Luca Martino,et al.  Physics-aware Gaussian processes in remote sensing , 2018, Appl. Soft Comput..

[252]  Steven L. Brunton,et al.  Inferring Biological Networks by Sparse Identification of Nonlinear Dynamics , 2016, IEEE Transactions on Molecular, Biological and Multi-Scale Communications.

[253]  Omri Azencot,et al.  Forecasting Sequential Data using Consistent Koopman Autoencoders , 2020, ICML.

[254]  David Galbally,et al.  Non‐linear model reduction for uncertainty quantification in large‐scale inverse problems , 2009 .

[255]  E Weinan,et al.  Deep Learning-Based Numerical Methods for High-Dimensional Parabolic Partial Differential Equations and Backward Stochastic Differential Equations , 2017, Communications in Mathematics and Statistics.

[256]  Mohammad M. Sultan,et al.  Transferable Neural Networks for Enhanced Sampling of Protein Dynamics. , 2018, Journal of chemical theory and computation.

[257]  Birgit Kirsch,et al.  Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems , 2019 .

[258]  Ilias Bilionis,et al.  Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification , 2018, J. Comput. Phys..

[259]  Erica M. Rutter,et al.  Learning partial differential equations for biological transport models from noisy spatio-temporal data , 2019, Proceedings of the Royal Society A.

[260]  Ramesh Raskar,et al.  Designing Neural Network Architectures using Reinforcement Learning , 2016, ICLR.

[261]  J. Templeton,et al.  Reynolds averaged turbulence modelling using deep neural networks with embedded invariance , 2016, Journal of Fluid Mechanics.

[262]  Pierre Baldi,et al.  Enforcing Analytic Constraints in Neural Networks Emulating Physical Systems. , 2019, Physical review letters.

[263]  Lexing Ying,et al.  A Multiscale Neural Network Based on Hierarchical Matrices , 2018, Multiscale Model. Simul..

[264]  T. Belytschko,et al.  A first course in finite elements , 2007 .

[265]  Pierre Gentine,et al.  Achieving Conservation of Energy in Neural Network Emulators for Climate Modeling , 2019, ArXiv.

[266]  P. Baldi,et al.  Jet Substructure Classification in High-Energy Physics with Deep Neural Networks , 2016, 1603.09349.

[267]  Pierre Gentine,et al.  Could Machine Learning Break the Convection Parameterization Deadlock? , 2018, Geophysical Research Letters.

[268]  Anuj Karpatne,et al.  Physics-Guided Machine Learning for Scientific Discovery: An Application in Simulating Lake Temperature Profiles , 2020, Trans. Data Sci..

[269]  Chao Hu,et al.  Physics-Based Convolutional Neural Network for Fault Diagnosis of Rolling Element Bearings , 2019, IEEE Sensors Journal.

[270]  Ed H. Chi,et al.  AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networks , 2019, ICLR.

[271]  Maziar Raissi,et al.  Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations , 2018, J. Mach. Learn. Res..

[272]  David W Toth,et al.  The TensorMol-0.1 model chemistry: a neural network augmented with long-range physics , 2017, Chemical science.

[273]  E Weinan,et al.  Machine Learning Approximation Algorithms for High-Dimensional Fully Nonlinear Partial Differential Equations and Second-order Backward Stochastic Differential Equations , 2017, J. Nonlinear Sci..

[274]  R Komanduri,et al.  Ab initio potential-energy surfaces for complex, multichannel systems using modified novelty sampling and feedforward neural networks. , 2005, The Journal of chemical physics.

[275]  A. Mohan,et al.  A Deep Learning based Approach to Reduced Order Modeling for Turbulent Flow Control using LSTM Neural Networks , 2018, 1804.09269.

[276]  Soumya Kundu,et al.  Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systems , 2017, 2019 American Control Conference (ACC).

[277]  Nicola De Cao,et al.  MolGAN: An implicit generative model for small molecular graphs , 2018, ArXiv.

[278]  Albert J. Valocchi,et al.  Data-driven methods to improve baseflow prediction of a regional groundwater model , 2015, Comput. Geosci..

[279]  Ankush Khandelwal,et al.  Bringing automated, remote‐sensed, machine learning methods to monitoring crop landscapes at scale , 2019, Agricultural Economics.

[280]  Pierre Baldi,et al.  Synergies Between Quantum Mechanics and Machine Learning in Reaction Prediction , 2016, J. Chem. Inf. Model..

[281]  Wojciech Samek,et al.  Learning the invisible: a hybrid deep learning-shearlet framework for limited angle computed tomography , 2018, Inverse Problems.

[282]  Ganix Esnaola,et al.  Short-term forecasting of the wave energy flux: Analogues, random forests, and physics-based models , 2015 .

[283]  S. De,et al.  Integrating machine learning and multiscale modeling—perspectives, challenges, and opportunities in the biological, biomedical, and behavioral sciences , 2019, npj Digital Medicine.

[284]  Jianyu Zhang,et al.  Symplectic Recurrent Neural Networks , 2020, ICLR.

[285]  Thomas Bolton,et al.  Applications of Deep Learning to Ocean Data Inference and Subgrid Parameterization , 2019, Journal of Advances in Modeling Earth Systems.

[286]  Carola-Bibiane Schönlieb,et al.  Adversarial Regularizers in Inverse Problems , 2018, NeurIPS.

[287]  Silvia Ferrari,et al.  A constrained integration (CINT) approach to solving partial differential equations using artificial neural networks , 2015, Neurocomputing.

[288]  Habib N. Najm,et al.  Workshop Report on Basic Research Needs for Scientific Machine Learning: Core Technologies for Artificial Intelligence , 2018 .

[289]  Paris Perdikaris,et al.  Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data , 2019, J. Comput. Phys..

[290]  R. Coifman,et al.  Fast wavelet transforms and numerical algorithms I , 1991 .

[291]  Fabien Casenave,et al.  Model order reduction assisted by deep neural networks (ROM-net) , 2020, Advanced Modeling and Simulation in Engineering Sciences.

[292]  Anuj Karpatne,et al.  Physics Guided RNNs for Modeling Dynamical Systems: A Case Study in Simulating Lake Temperature Profiles , 2018, SDM.

[293]  Kevin M. Ryan,et al.  Crystal Structure Prediction via Deep Learning. , 2018, Journal of the American Chemical Society.

[294]  S. Gershman.  Empirical priors for reinforcement learning models , 2016 .

[295]  Yike Guo,et al.  A reduced order model for turbulent flows in the urban environment using machine learning , 2019, Building and Environment.

[296]  Nikhil Muralidhar,et al.  Incorporating Prior Domain Knowledge into Deep Neural Networks , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[297]  Lexing Ying,et al.  Solving Electrical Impedance Tomography with Deep Learning , 2019, J. Comput. Phys..

[298]  Jian Sun,et al.  A theory-guided deep-learning formulation and optimization of seismic waveform inversion , 2020 .

[299]  Sergei Manzhos,et al.  A random-sampling high dimensional model representation neural network for building potential energy surfaces. , 2006, The Journal of chemical physics.

[300]  Ben Glocker,et al.  Distance Metric Learning Using Graph Convolutional Networks: Application to Functional Brain Networks , 2017, MICCAI.

[301]  Chengqi Zhang,et al.  Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , 2015, KDD.

[302]  Chris L. Pettit,et al.  Uncertainty Quantification in Aeroelasticity: Recent Results and Research Challenges , 2004 .

[303]  Alex Fout,et al.  Protein Interface Prediction using Graph Convolutional Networks , 2017, NIPS.

[304]  Jinlong Wu,et al.  Physics-informed machine learning approach for augmenting turbulence models: A comprehensive framework , 2018, Physical Review Fluids.