Integrating Machine Learning with Physics-Based Modeling

Machine learning is poised as a very powerful tool that can drastically improve our ability to carry out scientific research. However, many issues need to be addressed before this becomes a reality. This article focuses on one particular issue of broad interest: How can we integrate machine learning with physics-based modeling to develop new interpretable and truly reliable physical models? After introducing the general guidelines, we discuss the two most important issues for developing machine learning-based physical models: Imposing physical constraints and obtaining optimal datasets. We also provide a simple and intuitive explanation for the fundamental reasons behind the success of modern machine learning, as well as an introduction to the concurrent machine learning framework needed for integrating machine learning with physics-based modeling. Molecular dynamics and moment closure of kinetic equations are used as examples to illustrate the main issues discussed. We end with a general discussion on where this integration will lead us to, and where the new frontier will be after machine learning is successfully integrated into scientific modeling.

[1]  F. Sarghini,et al.  Neural networks based subgrid scale modeling in large eddy simulations , 2003 .

[2]  Weber,et al.  Computer simulation of local order in condensed phases of silicon. , 1985, Physical review. B, Condensed matter.

[3]  W. Kohn,et al.  Self-Consistent Equations Including Exchange and Correlation Effects , 1965 .

[4]  Michele Parrinello,et al.  Generalized neural-network representation of high-dimensional potential-energy surfaces. , 2007, Physical review letters.

[5]  E. Weinan,et al.  Ground State Energy Functional with Hartree-Fock Efficiency and Chemical Accuracy. , 2020, The journal of physical chemistry. A.

[6]  Chao Ma,et al.  Uniformly accurate machine learning-based hydrodynamic models for kinetic equations , 2019, Proceedings of the National Academy of Sciences.

[7]  Adrian E. Roitberg,et al.  Less is more: sampling chemical space with active learning , 2018, The Journal of chemical physics.

[8]  R. Hentschke.  Non-Equilibrium Thermodynamics , 2014 .

[9]  Alexander J. Smola,et al.  Deep Sets , 2017, 1703.06114.

[10]  P. Dirac.  Quantum Mechanics of Many-Electron Systems , 1929 .

[11]  L. Reatto,et al.  The Ground State of Liquid He(4) , 1969 .

[12]  Linfeng Zhang,et al.  DeePCG: Constructing coarse-grained models via deep neural networks. , 2018, The Journal of chemical physics.

[13]  E Weinan,et al.  Pushing the Limit of Molecular Dynamics with Ab Initio Accuracy to 100 Million Atoms with Machine Learning , 2020, SC20: International Conference for High Performance Computing, Networking, Storage and Analysis.

[14]  R. Kondor,et al.  Gaussian approximation potentials: the accuracy of quantum mechanics, without the electrons. , 2009, Physical review letters.

[15]  E Weinan,et al.  Reinforced dynamics for enhanced sampling in large atomic and molecular systems. I. Basic Methodology , 2017, The Journal of chemical physics.

[16]  E Weinan,et al.  End-to-end Symmetry Preserving Inter-atomic Potential Energy Model for Finite and Extended Systems , 2018, NeurIPS.

[17]  R. Car,et al.  Raman spectrum and polarizability of liquid water from deep neural networks. , 2020, Physical chemistry chemical physics : PCCP.

[18]  Ryo Nagai,et al.  Completing density functional theory by machine learning hidden messages from molecules , 2019, npj Computational Materials.

[19]  E Weinan,et al.  Active Learning of Uniformly Accurate Inter-atomic Potentials for Materials Simulation , 2018, Physical Review Materials.

[20]  Noam Bernstein,et al.  De novo exploration and self-guided learning of potential-energy surfaces , 2019, npj Computational Materials.

[21]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[22]  H. Weyl.  The Classical Groups , 1940 .

[23]  Weinan E,et al.  Deep neural network for the dielectric response of insulators , 2020 .

[24]  R. Kondor,et al.  On representing chemical environments , 2012, 1209.3140.

[25]  E Weinan,et al.  Solving many-electron Schrödinger equation using deep neural networks , 2018, J. Comput. Phys..

[26]  Car,et al.  Unified approach for molecular dynamics and density-functional theory. , 1985, Physical review letters.

[27]  J. Lumley,et al.  A First Course in Turbulence , 1972 .

[28]  Nicholas Kevlahan,et al.  Principles of Multiscale Modeling , 2012 .

[29]  E Weinan,et al.  DP-GEN: A concurrent learning platform for the generation of reliable deep learning based potential energy models , 2019, Comput. Phys. Commun..

[30]  P. Bhatnagar,et al.  A Model for Collision Processes in Gases. I. Small Amplitude Processes in Charged and Neutral One-Component Systems , 1954 .

[31]  Alexander V. Shapeev,et al.  Active learning of linearly parametrized interatomic potentials , 2016, 1611.09346.

[32]  E Weinan,et al.  Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanics , 2017, Physical review letters.

[33]  E Weinan,et al.  86 PFLOPS Deep Potential Molecular Dynamics simulation of 100 million atoms with ab initio accuracy , 2020, Comput. Phys. Commun..

[34]  E Weinan,et al.  The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems , 2017, Communications in Mathematics and Statistics.

[35]  E Weinan,et al.  DeePMD-kit: A deep learning package for many-body potential energy representation and molecular dynamics , 2017, Comput. Phys. Commun..

[36]  Thomas F. Miller,et al.  A Universal Density Matrix Functional from Molecular Orbital-Based Machine Learning: Transferability across Organic Molecules , 2019, The Journal of chemical physics.

[37]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[38]  M. Yoneya,et al.  Physics of Liquid Crystals , 2014 .

[39]  D. Ceperley,et al.  Monte Carlo simulation of a many-fermion study , 1977 .

[40]  Matthias Troyer,et al.  Solving the quantum many-body problem with artificial neural networks , 2016, Science.

[41]  E. Weinan,et al.  Deep Potential: a general representation of a many-body potential energy surface , 2017, 1707.01478.

[42]  E Weinan,et al.  Model Reduction with Memory and the Machine Learning of Dynamical Systems , 2018, Communications in Computational Physics.

[43]  H. Grad.  On the kinetic theory of rarefied gases , 1949 .

[44]  E Weinan,et al.  Machine learning based non-Newtonian fluid model with molecular fidelity , 2020, Physical review. E.