OrbNet: Deep Learning for Quantum Chemistry Using Symmetry-Adapted Atomic-Orbital Features

We introduce a machine learning method in which energy solutions from the Schrödinger equation are predicted using symmetry adapted atomic orbital features and a graph neural-network architecture. OrbNet is shown to outperform existing methods in terms of learning efficiency and transferability for the prediction of density functional theory results while employing low-cost features that are obtained from semi-empirical electronic structure calculations. For applications to datasets of drug-like molecules, including QM7b-T, QM9, GDB-13-T, DrugBank, and the conformer benchmark dataset of Folmsbee and Hutchison [Int. J. Quantum Chem. (published online) (2020)], OrbNet predicts energies within chemical accuracy of density functional theory at a computational cost that is 1000-fold or more reduced.

[1]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[2]  Stephan Günnemann,et al.  Directional Message Passing for Molecular Graphs , 2020, ICLR.

[3]  Nicholay Topin,et al.  Super-convergence: very fast training of neural networks using large learning rates , 2018, Defense + Commercial Sensing.

[4]  R. Kondor,et al.  Gaussian approximation potentials: the accuracy of quantum mechanics, without the electrons. , 2009, Physical review letters.

[5]  Frederick R. Manby,et al.  Fast Hartree–Fock theory using local density fitting approximations , 2004 .

[6]  Thomas F. Miller,et al.  Regression-clustering for Improved Accuracy and Training Cost with Molecular-Orbital-Based Machine Learning , 2019, Journal of chemical theory and computation.

[7]  Vijay S. Pande,et al.  Molecular graph convolutions: moving beyond fingerprints , 2016, Journal of Computer-Aided Molecular Design.

[8]  E Weinan,et al.  Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanics , 2017, Physical review letters.

[9]  Alexandre Tkatchenko,et al.  Quantum-chemical insights from deep tensor neural networks , 2016, Nature Communications.

[10]  A. Becke.  Density-functional thermochemistry. III. The role of exact exchange , 1993 .

[11]  Andrew G. Taube,et al.  Improving the accuracy of Møller-Plesset perturbation theory with neural networks. , 2017, The Journal of chemical physics.

[12]  M. Frisch,et al.  Ab Initio Calculation of Vibrational Absorption and Circular Dichroism Spectra Using Density Functional Force Fields , 1994 .

[13]  Michele Ceriotti,et al.  Recognizing molecular patterns by machine learning: an agnostic structural definition of the hydrogen bond. , 2014, The Journal of chemical physics.

[14]  Klaus-Robert Müller,et al.  SchNet: A continuous-filter convolutional neural network for modeling quantum interactions , 2017, NIPS.

[15]  Stefan Grimme,et al.  Ultra-fast computation of electronic spectra for large systems by tight-binding based simplified Tamm-Dancoff approximation (sTDA-xTB). , 2016, The Journal of chemical physics.

[16]  Jörg Behler,et al.  Comparison of permutationally invariant polynomials, neural networks, and Gaussian approximation potentials in representing water interactions through many-body expansions. , 2018, The Journal of chemical physics.

[17]  Stefan Grimme,et al.  A Robust and Accurate Tight-Binding Quantum Chemical Method for Structures, Vibrational Frequencies, and Noncovalent Interactions of Large Molecular Systems Parametrized for All spd-Block Elements (Z = 1-86). , 2017, Journal of chemical theory and computation.

[18]  Volker L. Deringer,et al.  Gaussian approximation potential modeling of lithium intercalation in carbon nanostructures. , 2017, The Journal of chemical physics.

[19]  Regina Barzilay,et al.  Analyzing Learned Molecular Representations for Property Prediction , 2019, J. Chem. Inf. Model..

[20]  K. Müller,et al.  Fast and accurate modeling of molecular atomization energies with machine learning. , 2011, Physical review letters.

[21]  Matthias Rupp,et al.  Big Data Meets Quantum Chemistry Approximations: The Δ-Machine Learning Approach. , 2015, Journal of chemical theory and computation.

[22]  F. Weigend,et al.  Balanced basis sets of split valence, triple zeta valence and quadruple zeta valence quality for H to Rn: Design and assessment of accuracy. , 2005, Physical chemistry chemical physics : PCCP.

[23]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[24]  Parr,et al.  Development of the Colle-Salvetti correlation-energy formula into a functional of the electron density. , 1988, Physical review. B, Condensed matter.

[25]  Justin S. Smith,et al.  Hierarchical modeling of molecular energies using a deep neural network. , 2017, The Journal of chemical physics.

[26]  J. Behler.  Perspective: Machine learning potentials for atomistic simulations. , 2016, The Journal of chemical physics.

[27]  Karsten Reuter,et al.  Making the Coupled Cluster Correlation Energy Machine-Learnable. , 2018, The journal of physical chemistry. A.

[28]  Daniel G A Smith,et al.  Psi4 1.4: Open-source software for high-throughput quantum chemistry. , 2020, The Journal of chemical physics.

[29]  Lorenz C. Blum,et al.  970 million druglike small molecules for virtual screening in the chemical universe database GDB-13. , 2009, Journal of the American Chemical Society.

[30]  Jeng-Da Chai,et al.  Long-Range Corrected Hybrid Density Functionals with Improved Dispersion Corrections. , 2012, Journal of chemical theory and computation.

[31]  Geoffrey J. Gordon,et al.  A Density Functional Tight Binding Layer for Deep Learning of Chemical Hamiltonians. , 2018, Journal of chemical theory and computation.

[32]  Pavlo O. Dral,et al.  Quantum chemistry structures and properties of 134 kilo molecules , 2014, Scientific Data.

[33]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[34]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[35]  David W Toth,et al.  The TensorMol-0.1 model chemistry: a neural network augmented with long-range physics , 2017, Chemical science.

[36]  P. C. Hariharan,et al.  The influence of polarization functions on molecular orbital hydrogenation energies , 1973 .

[37]  Andreas Hansen,et al.  Excited states using the simplified Tamm-Dancoff-Approach for range-separated hybrid density functionals: development and application. , 2014, Physical chemistry chemical physics : PCCP.

[38]  Alberto Fabrizio,et al.  Transferable Machine-Learning Model of the Electron Density , 2018, ACS central science.

[39]  Thomas F. Miller,et al.  A Universal Density Matrix Functional from Molecular Orbital-Based Machine Learning: Transferability across Organic Molecules , 2019, The Journal of chemical physics.

[40]  Sebastian Dick,et al.  Machine learning accurate exchange and correlation functionals of the electronic density , 2020, Nature Communications.

[41]  M. Parrinello,et al.  Accurate sampling using Langevin dynamics. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[42]  Anders S. Christensen,et al.  Operator Quantum Machine Learning: Navigating the Chemical Space of Response Properties. , 2019, Chimia.

[43]  M. Rupp,et al.  Machine learning of molecular electronic properties in chemical compound space , 2013, 1305.7074.

[44]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[45]  Nicholas Lubbers,et al.  GPU-Accelerated Semi-Empirical Born Oppenheimer Molecular Dynamics using PyTorch. , 2020, Journal of chemical theory and computation.

[46]  S. H. Vosko,et al.  Accurate spin-dependent electron liquid correlation energies for local spin density calculations: a critical analysis , 1980 .

[47]  Stefan Grimme,et al.  GFN2-xTB-An Accurate and Broadly Parametrized Self-Consistent Tight-Binding Quantum Chemical Method with Multipole Electrostatics and Density-Dependent Dispersion Contributions. , 2018, Journal of Chemical Theory and Computation.

[48]  David S. Wishart,et al.  DrugBank 4.0: shedding new light on drug metabolism , 2013, Nucleic Acids Res..

[49]  Sicun Gao,et al.  Active learning of many-body configuration space: Application to the Cs+-water MB-nrg potential energy function as a case study. , 2020, The Journal of chemical physics.

[50]  Regina Barzilay,et al.  Correction to Analyzing Learned Molecular Representations for Property Prediction , 2019, J. Chem. Inf. Model..

[51]  Michele Parrinello,et al.  Generalized neural-network representation of high-dimensional potential-energy surfaces. , 2007, Physical review letters.

[52]  E. Weinan,et al.  Ground State Energy Functional with Hartree-Fock Efficiency and Chemical Accuracy. , 2020, The journal of physical chemistry. A.

[53]  Florian Weigend,et al.  Hartree–Fock exchange fitting basis sets for H to Rn † , 2008, J. Comput. Chem..

[54]  Anders S Christensen,et al.  FCHL revisited: Faster and more accurate quantum machine learning. , 2020, The Journal of chemical physics.

[55]  Vijay S. Pande,et al.  MoleculeNet: a benchmark for molecular machine learning , 2017, Chemical science.

[56]  J S Smith,et al.  ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost , 2016, Chemical science.

[57]  Kipton Barros,et al.  Approaching coupled cluster accuracy with a general-purpose neural network potential through transfer learning , 2019, Nature Communications.

[58]  Thomas F. Miller,et al.  Transferability in Machine Learning for Electronic Structure via the Molecular Orbital Basis. , 2018, Journal of chemical theory and computation.

[59]  Klaus-Robert Müller,et al.  Assessment and Validation of Machine Learning Methods for Predicting Molecular Atomization Energies. , 2013, Journal of chemical theory and computation.

[60]  Thomas F. Miller,et al.  Small Nuclear Quantum Effects in Scattering of H and D from Graphene. , 2021, The journal of physical chemistry letters.

[61]  Li Li,et al.  Bypassing the Kohn-Sham equations with machine learning , 2016, Nature Communications.

[62]  Markus Meuwly,et al.  PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments, and Partial Charges. , 2019, Journal of chemical theory and computation.

[63]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[64]  Stefan Grimme,et al.  A simplified Tamm-Dancoff density functional approach for the electronic excitation spectra of very large molecules. , 2013, The Journal of chemical physics.

[65]  Geoffrey Hutchison,et al.  Assessing Conformer Energies using Electronic Structure and Machine Learning Methods , 2020 .