Differentiable sampling of molecular geometries with uncertainty-based adversarial attacks

Neural network (NN) interatomic potentials provide fast prediction of potential energy surfaces, closely matching the accuracy of the electronic structure methods used to produce the training data. However, NN predictions are only reliable within well-learned training domains, and show volatile behavior when extrapolating. Uncertainty quantification methods can flag atomic configurations for which prediction confidence is low, but arriving at such uncertain regions requires expensive sampling of the NN phase space, often using atomistic simulations. Here, we exploit automatic differentiation to drive atomistic systems towards high-likelihood, high-uncertainty configurations without the need for molecular dynamics simulations. By performing adversarial attacks on an uncertainty metric, informative geometries that expand the training domain of NNs are sampled. When combined with an active learning loop, this approach bootstraps and improves NN potentials while decreasing the number of calls to the ground truth method. This efficiency is demonstrated on sampling of kinetic barriers, collective variables in molecules, and supramolecular chemistry in zeolite-molecule interactions, and can be extended to any NN potential architecture and materials system.

[1]  H. Monkhorst,et al.  SPECIAL POINTS FOR BRILLOUIN-ZONE INTEGRATIONS , 1976 .

[2]  R. Kondor,et al.  On representing chemical environments , 2012, 1209.3140.

[3]  G. Kresse,et al.  Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set , 1996 .

[4]  Yan Zhao,et al.  Exchange-correlation functional with broad accuracy for metallic and nonmetallic compounds, kinetics, and noncovalent interactions. , 2005, The Journal of chemical physics.

[5]  Tim Mueller,et al.  Machine learning for interatomic potential models. , 2020, The Journal of chemical physics.

[6]  S. Grimme,et al.  A consistent and accurate ab initio parametrization of density functional dispersion correction (DFT-D) for the 94 elements H-Pu. , 2010, The Journal of chemical physics.

[7]  Joseph P. Romano,et al.  Large Sample Confidence Regions Based on Subsamples under Minimal Assumptions , 1994 .

[8]  Filippo Federici Canova,et al.  DScribe: Library of Descriptors for Machine Learning in Materials Science , 2019, Comput. Phys. Commun..

[9]  T. Halgren Merck molecular force field. I. Basis, form, scope, parameterization, and performance of MMFF94 , 1996, J. Comput. Chem..

[10]  A. Zunger Inverse design in search of materials with target functionalities , 2018 .

[11]  A. Tkatchenko,et al.  Challenges for machine learning force fields in reproducing potential energy surfaces of flexible molecules. , 2021, The Journal of chemical physics.

[12]  Daniel W. Davies,et al.  Machine learning for molecular and materials science , 2018, Nature.

[13]  Noam Bernstein,et al.  Machine learning unifies the modeling of materials and molecules , 2017, Science Advances.

[14]  J S Smith,et al.  ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost , 2016, Chemical science.

[15]  Burke,et al.  Generalized Gradient Approximation Made Simple. , 1996, Physical review letters.

[16]  Wenwu Zhu,et al.  Robust Graph Convolutional Networks Against Adversarial Attacks , 2019, KDD.

[17]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[18]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[19]  Rafael Gómez-Bombarelli,et al.  Generative Models for Automatic Chemical Design , 2019, Machine Learning Meets Quantum Physics.

[20]  K-R Müller,et al.  SchNet - A deep learning architecture for molecules and materials. , 2017, The Journal of chemical physics.

[21]  Kresse,et al.  Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set. , 1996, Physical review. B, Condensed matter.

[22]  Lei Chen,et al.  Iterative training set refinement enables reactive molecular dynamics via machine learned forces , 2020 .

[23]  A. Becke,et al.  Density-functional exchange-energy approximation with correct asymptotic behavior. , 1988, Physical review. A, General physics.

[24]  Stephan Günnemann,et al.  Directional Message Passing for Molecular Graphs , 2020, ICLR.

[25]  M. Ceriotti,et al.  Evidence for supercritical behaviour of high-pressure liquid hydrogen , 2019, Nature.

[26]  K. Müller,et al.  Towards exact molecular dynamics simulations with machine-learned force fields , 2018, Nature Communications.

[27]  Nongnuch Artrith,et al.  An implementation of artificial neural-network potentials for atomistic materials simulations: Performance for TiO2 , 2016 .

[28]  William L. Jorgensen,et al.  LigParGen web server: an automatic OPLS-AA parameter generator for organic ligands , 2017, Nucleic Acids Res..

[29]  Rafael Gómez-Bombarelli,et al.  Benchmarking binding energy calculations for organic structure-directing agents in pure-silica zeolites. , 2021, The Journal of chemical physics.

[30]  J. Behler Neural network potential-energy surfaces in chemistry: a tool for large-scale simulations. , 2011, Physical chemistry chemical physics : PCCP.

[31]  Michele Ceriotti,et al.  Uncertainty estimation for molecular dynamics and sampling. , 2021, The Journal of chemical physics.

[32]  R. Clemen Combining forecasts: A review and annotated bibliography , 1989 .

[33]  Michele Parrinello,et al.  Nucleation mechanism for the direct graphite-to-diamond phase transition. , 2011, Nature materials.

[34]  Alireza Khorshidi,et al.  Addressing uncertainty in atomistic machine learning. , 2017, Physical chemistry chemical physics : PCCP.

[35]  Pekka Koskinen,et al.  Structural relaxation made simple. , 2006, Physical review letters.

[36]  Thomas Bligaard,et al.  Low-Scaling Algorithm for Nudged Elastic Band Calculations Using a Surrogate Machine Learning Model. , 2018, Physical review letters.

[37]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[38]  T. Morawietz,et al.  How van der Waals interactions determine the unique properties of water , 2016, Proceedings of the National Academy of Sciences.

[39]  Alexander V. Shapeev,et al.  Active Learning and Uncertainty Estimation , 2020 .

[40]  Frank Neese,et al.  Software update: the ORCA program system, version 4.0 , 2018 .

[41]  Ondrej Marsalek,et al.  Committee neural network potentials control generalization errors and enable active learning , 2020, The Journal of chemical physics.

[42]  Haitao Liu,et al.  When Gaussian Process Meets Big Data: A Review of Scalable GPs , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[43]  Shi Jun Ang,et al.  Active learning accelerates ab initio molecular dynamics on reactive energy surfaces , 2021, Chem.

[44]  Ken-ichi Kawarabayashi,et al.  How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks , 2020, ICLR.

[45]  Julien Cornebise,et al.  Weight Uncertainty in Neural Network , 2015, ICML.

[46]  Georg Kresse,et al.  Phase Transitions of Hybrid Perovskites Simulated by Machine-Learning Force Fields Trained on the Fly with Bayesian Inference. , 2019, Physical review letters.

[47]  A. Laio,et al.  Escaping free-energy minima , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[48]  G. Kresse,et al.  From ultrasoft pseudopotentials to the projector augmented-wave method , 1999 .

[49]  G. Henkelman,et al.  A climbing image nudged elastic band method for finding saddle points and minimum energy paths , 2000 .

[50]  Blöchl,et al.  Projector augmented-wave method. , 1994, Physical review. B, Condensed matter.

[51]  Thomas A. Halgren Merck molecular force field. I. Basis, form, scope, parameterization, and performance of MMFF94 , 1996, J. Comput. Chem..

[52]  Rampi Ramprasad,et al.  Machine Learning Force Fields: Construction, Validation, and Outlook , 2016, 1610.02098.

[53]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[54]  Yaolong Zhang,et al.  Automatically growing global reactive neural network potential energy surfaces: A trajectory-free active learning strategy. , 2020, The Journal of chemical physics.

[55]  Aleksander Madry,et al.  Robustness May Be at Odds with Accuracy , 2018, ICLR.

[56]  Frank Noé,et al.  Targeted Adversarial Learning Optimized Sampling. , 2019, The journal of physical chemistry letters.

[57]  Nancy Wilkins-Diehr,et al.  XSEDE: Accelerating Scientific Discovery , 2014, Computing in Science & Engineering.

[58]  Rafael Gómez-Bombarelli,et al.  Supramolecular Recognition in Crystalline Nanocavities through Monte Carlo and Voronoi Network Algorithms , 2021 .

[59]  Felix Hill,et al.  Measuring abstract reasoning in neural networks , 2018, ICML.

[60]  J. Perdew,et al.  Density-functional approximation for the correlation energy of the inhomogeneous electron gas. , 1986, Physical review. B, Condensed matter.

[61]  M. Gastegger,et al.  High-Dimensional Neural Network Potentials for Organic Reactions and an Improved Training Algorithm. , 2015, Journal of chemical theory and computation.

[62]  Hao Wu,et al.  Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning , 2018, Science.

[63]  Stephan Günnemann,et al.  Adversarial Attacks on Neural Networks for Graph Data , 2018, KDD.

[64]  Vijay S. Pande,et al.  OpenMM 7: Rapid development of high performance algorithms for molecular dynamics , 2016, bioRxiv.

[65]  Jörg Behler,et al.  Neural network molecular dynamics simulations of solid-liquid interfaces: water at low-index copper surfaces. , 2016, Physical chemistry chemical physics : PCCP.

[66]  Stefan Grimme,et al.  Effect of the damping function in dispersion corrected density functional theory , 2011, J. Comput. Chem..

[67]  J. Behler,et al.  Construction of high-dimensional neural network potentials using environment-dependent atom pairs. , 2012, The Journal of chemical physics.

[68]  Jonathan Vandermause,et al.  On-the-fly active learning of interpretable Bayesian force fields for atomistic rare events , 2020, npj Computational Materials.

[69]  Leland McInnes,et al.  UMAP: Uniform Manifold Approximation and Projection , 2018, J. Open Source Softw..

[70]  Rafael Gómez-Bombarelli,et al.  Active learning and neural network potentials accelerate molecular screening of ether-based solvate ionic liquids. , 2020, Chemical communications.

[71]  Vijay S. Pande,et al.  Accelerating molecular dynamic simulation on graphics processing units , 2009, J. Comput. Chem..

[72]  Michael Walter,et al.  The atomic simulation environment-a Python library for working with atoms. , 2017, Journal of physics. Condensed matter : an Institute of Physics journal.

[73]  W. L. Jorgensen,et al.  Improved Peptide and Protein Torsional Energetics with the OPLS-AA Force Field , 2015, Journal of chemical theory and computation.

[74]  E Weinan,et al.  Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanics , 2017, Physical review letters.

[75]  H. Jónsson,et al.  Nudged elastic band method for finding minimum energy paths of transitions , 1998 .

[76]  Michele Parrinello,et al.  Generalized neural-network representation of high-dimensional potential-energy surfaces. , 2007, Physical review letters.

[77]  R. Jaffe,et al.  A Bayesian Machine Learning Approach to the Quantification of Uncertainties on Ab Initio Potential Energy Surfaces. , 2020, The journal of physical chemistry. A.

[78]  Ian J. Goodfellow,et al.  Adversarial Forces of Physical Models , 2020 .