A fast neural network approach for direct covariant forces prediction in complex multi-element extended systems

Neural network force field (NNFF) is a method for performing regression on atomic structure–force relationships, bypassing the expensive quantum mechanics calculations that prevent the execution of long ab initio quality molecular dynamics (MD) simulations. However, most NNFF methods for complex multi-element atomic systems indirectly predict atomic force vectors by exploiting just atomic structure rotation-invariant features and network-feature spatial derivatives, which are computationally expensive. Here, we show a staggered NNFF architecture that exploits both rotation-invariant and -covariant features to directly predict atomic force vectors without using spatial derivatives, and we demonstrate 2.2× NNFF–MD acceleration over a state-of-the-art C++ engine using a Python engine. This fast architecture enables us to develop NNFF for complex ternary- and quaternary-element extended systems composed of long polymer chains, amorphous oxide and surface chemical reactions. The rotation-invariant–covariant architecture described here can also directly predict complex covariant vector outputs from local environments, in other domains beyond computational material science.Neural network force fields promise to bypass the computationally expensive quantum mechanical calculations typically required to investigate complex materials, such as lithium-ion batteries. Mailoa et al. accelerate these approaches with an architecture that exploits both rotation-invariant and -covariant features separately.

[1]  Jim Pfaendtner,et al.  Car–Parrinello Molecular Dynamics + Metadynamics Study of High-Temperature Methanol Oxidation Reactions Using Generic Collective Variables , 2014 .

[2]  Rampi Ramprasad,et al.  A universal strategy for the creation of machine learning-based atomistic force fields , 2017, npj Computational Materials.

[3]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[4]  Alfonso Pedone,et al.  A new self-consistent empirical interatomic potential model for oxides, silicates, and silica-based glasses. , 2006, The journal of physical chemistry. B.

[5]  Alok Choudhary,et al.  Combinatorial screening for new materials in unconstrained composition space with machine learning , 2014 .

[6]  Michael Gastegger,et al.  Machine learning molecular dynamics for the simulation of infrared spectra† †Electronic supplementary information (ESI) available. See DOI: 10.1039/c7sc02267k , 2017, Chemical science.

[7]  Boris Merinov,et al.  Molecular Dynamics Simulations of Ionic Diffusion in PEO-LiTFSI Polymer Electrolyte: Effect of Temperature, Molecular Weight, and Ionic Concentration , 2018 .

[8]  Nongnuch Artrith,et al.  High-dimensional neural network potentials for metal surfaces: A prototype study for copper , 2012 .

[9]  James Zou,et al.  Feedback GAN for DNA optimizes protein functions , 2019, Nature Machine Intelligence.

[10]  Alexie M. Kolpak,et al.  Discovering charge density functionals and structure-property relationships with PROPhet: A general framework for coupling machine learning and first-principles methods , 2017, Scientific Reports.

[11]  Rajiv K. Kalia,et al.  Neural Network Analysis of Dynamic Fracture in a Layered Material , 2019, MRS Advances.

[12]  Li Li,et al.  Tensor Field Networks: Rotation- and Translation-Equivariant Neural Networks for 3D Point Clouds , 2018, ArXiv.

[13]  J S Smith,et al.  ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost , 2016, Chemical science.

[14]  E Banas,et al.  Search for Higgs and Z boson decays to J/ψγ and ϒ(nS)γ with the ATLAS detector , 2015, 1501.03276.

[15]  D. Sánchez-Portal,et al.  Numerical atomic orbitals for linear-scaling calculations , 2001, cond-mat/0104170.

[16]  John E Herr,et al.  Metadynamics for training neural network model chemistries: A competitive assessment. , 2017, The Journal of chemical physics.

[17]  W. Press,et al.  Numerical Recipes in C++: The Art of Scientific Computing (2nd edn)1 Numerical Recipes Example Book (C++) (2nd edn)2 Numerical Recipes Multi-Language Code CD ROM with LINUX or UNIX Single-Screen License Revised Version3 , 2003 .

[18]  R. C. Macridis.  A review , 1963 .

[19]  J. Behler.  Atom-centered symmetry functions for constructing high-dimensional neural network potentials. , 2011, The Journal of chemical physics.

[20]  G. Kresse,et al.  From ultrasoft pseudopotentials to the projector augmented-wave method , 1999 .

[21]  Edieal J. Pinker,et al.  Reporting accuracy of rare event classifiers , 2018, npj Digital Medicine.

[22]  Burke,et al.  Generalized Gradient Approximation Made Simple. , 1996, Physical review letters.

[23]  Bernard R Brooks,et al.  Machine Learning Force Field Parameters from Ab Initio Data. , 2017, Journal of chemical theory and computation.

[24]  Patrick Huck,et al.  Active learning for accelerated design of layered materials , 2018, npj Computational Materials.

[25]  R. Kondor,et al.  Gaussian approximation potentials: the accuracy of quantum mechanics, without the electrons. , 2009, Physical review letters.

[26]  K. Müller,et al.  Towards exact molecular dynamics simulations with machine-learned force fields , 2018, Nature Communications.

[27]  Nicola Gaston,et al.  Building machine learning force fields for nanoclusters. , 2018, The Journal of chemical physics.

[28]  Jonathan Vandermause,et al.  On-the-Fly Bayesian Active Learning of Interpretable Force-Fields for Atomistic Rare Events , 2019 .

[29]  J. Nathan Kutz,et al.  Deep learning in fluid dynamics , 2017, Journal of Fluid Mechanics.

[30]  Risi Kondor,et al.  N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials , 2018, ArXiv.

[31]  M. Gastegger,et al.  High-Dimensional Neural Network Potentials for Organic Reactions and an Improved Training Algorithm. , 2015, Journal of chemical theory and computation.

[32]  Alireza Khorshidi,et al.  Amp: A modular approach to machine learning in atomistic simulations , 2016, Comput. Phys. Commun..

[33]  K-R Müller,et al.  SchNet - A deep learning architecture for molecules and materials. , 2017, The Journal of chemical physics.

[34]  Kresse,et al.  Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set. , 1996, Physical review. B, Condensed matter.

[35]  Jeffrey C Grossman,et al.  Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties. , 2017, Physical review letters.

[36]  Allen M. Hermann,et al.  Bipolar plates for PEM fuel cells: A review , 2005 .

[37]  E Weinan,et al.  End-to-end Symmetry Preserving Inter-atomic Potential Energy Model for Finite and Extended Systems , 2018, NeurIPS.

[38]  Jörg Behler,et al.  Automatic selection of atomic fingerprints and reference configurations for machine-learning potentials. , 2018, The Journal of chemical physics.

[39]  G. Kresse,et al.  Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set , 1996 .

[40]  S. Nosé.  A unified formulation of the constant temperature molecular dynamics methods , 1984 .

[41]  A. Thompson,et al.  Computational aspects of many-body potentials , 2012 .

[42]  Boris Kozinsky,et al.  Effect of Salt Concentration on Ion Clustering and Transport in Polymer Solid Electrolytes: A Molecular Dynamics Study of PEO–LiTFSI , 2018, Chemistry of Materials.

[43]  R. McGibbon,et al.  Discovering chemistry with an ab initio nanoreactor , 2014, Nature chemistry.

[44]  William A. Goddard,et al.  Atomistic Description of Ionic Diffusion in PEO–LiTFSI: Effect of Temperature, Molecular Weight, and Ionic Concentration , 2018, Macromolecules.

[45]  Beatriz Cordero,et al.  Covalent radii revisited. , 2008, Dalton transactions.

[46]  Wei-keng Liao,et al.  ElemNet: Deep Learning the Chemistry of Materials From Only Elemental Composition , 2018, Scientific Reports.

[47]  Megan F. McCracken.  Artificial Neural Networks in Fluid Dynamics: A Novel Approach to the Navier-Stokes Equations , 2018, PEARC.

[48]  Pascal Friederich,et al.  Machine learning of correlated dihedral potentials for atomistic molecular force fields , 2018, Scientific Reports.

[49]  Zhenwei Li,et al.  Molecular dynamics with on-the-fly machine learning of quantum-mechanical forces. , 2015, Physical review letters.

[50]  Daniel W. Davies,et al.  Machine learning for molecular and materials science , 2018, Nature.

[51]  P. Popelier,et al.  Potential energy surfaces fitted by artificial neural networks. , 2010, The journal of physical chemistry. A.

[52]  William H. Press,et al.  Numerical recipes: the art of scientific computing, 3rd Edition , 2007 .

[53]  Turab Lookman,et al.  Developing an interatomic potential for martensitic phase transformations in zirconium by machine learning , 2018, npj Computational Materials.

[54]  Michael R Shirts,et al.  Testing for physical validity in molecular simulations , 2018, PloS one.

[55]  Satoshi Watanabe,et al.  Study of Li atom diffusion in amorphous Li3PO4 with neural network potential. , 2017, The Journal of chemical physics.

[56]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[57]  Steve Plimpton,et al.  Fast parallel algorithms for short-range molecular dynamics , 1993 .

[58]  G. Kresse,et al.  Ab initio molecular dynamics for liquid metals. , 1993 .

[59]  E. G. Novik,et al.  Reentrant topological phases in Mn-doped HgTe quantum wells , 2012, 1203.1491.

[60]  Peter Sollich,et al.  Accurate interatomic force fields via machine learning with covariant kernels , 2016, 1611.03877.

[61]  A. V. Duin,et al.  ReaxFF: A Reactive Force Field for Hydrocarbons , 2001 .

[62]  Jonathan Vandermause,et al.  On-the-fly active learning of interpretable Bayesian force fields for atomistic rare events , 2020, npj Computational Materials.

[63]  E Weinan,et al.  Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanics , 2017, Physical review letters.

[64]  Klaus-Robert Müller,et al.  Machine learning of accurate energy-conserving molecular force fields , 2016, Science Advances.

[65]  Michele Parrinello,et al.  Generalized neural-network representation of high-dimensional potential-energy surfaces. , 2007, Physical review letters.

[66]  Kwang-Tzu Yang,et al.  Artificial Neural Networks (ANNs) : A New Paradigm for Thermal Science and Engineering , 2008 .

[67]  Richard A. Friesner,et al.  Integrated Modeling Program, Applied Chemical Theory (IMPACT) , 2005, J. Comput. Chem..