Uncertainty Estimation for Molecules: Desiderata and Methods

Graph Neural Networks (GNNs) are promising surrogates for quantum mechanical calculations as they establish unprecedented low errors on collections of molecular dynamics (MD) trajectories. Thanks to their fast inference times they promise to accelerate computational chemistry applications. Unfortunately, despite low in-distribution (ID) errors, such GNNs might be horribly wrong for out-of-distribution (OOD) samples. Uncertainty estimation (UE) may aid in such situations by communicating the model's certainty about its prediction. Here, we take a closer look at the problem and identify six key desiderata for UE in molecular force fields, three 'physics-informed' and three 'application-focused' ones. To overview the field, we survey existing methods from the field of UE and analyze how they fit to the set desiderata. By our analysis, we conclude that none of the previous works satisfies all criteria. To fill this gap, we propose Localized Neural Kernel (LNK) a Gaussian Process (GP)-based extension to existing GNNs satisfying the desiderata. In our extensive experimental evaluation, we test four different UE with three different backbones and two datasets. In out-of-equilibrium detection, we find LNK yielding up to 2.5 and 2.1 times lower errors in terms of AUC-ROC score than dropout or evidential regression-based methods while maintaining high predictive performance.

[1]  Stephan Gunnemann,et al.  Generalizing Neural Wave Functions , 2023, ICML.

[2]  Zachary W. Ulissi,et al.  AdsorbML: Accelerating Adsorption Energy Calculations with Machine Learning , 2022, ArXiv.

[3]  Jan Strohbeck,et al.  Deep Kernel Learning for Uncertainty Estimation in Multiple Trajectory Prediction Networks , 2022, 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[4]  T. Jaakkola,et al.  Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations , 2022, ArXiv.

[5]  Johannes T. Margraf,et al.  How robust are modern graph neural network potentials in long and hot molecular dynamics simulations? , 2022, Mach. Learn. Sci. Technol..

[6]  Stephan Gunnemann,et al.  Sampling-free Inference for Ab-Initio Potential Energy Surface Networks , 2022, ICLR.

[7]  Simon L. Batzner,et al.  The Design Space of E(3)-Equivariant Atom-Centered Interatomic Potentials , 2022, ArXiv.

[8]  Wenwu Zhu,et al.  Out-Of-Distribution Generalization on Graphs: A Survey , 2022, ArXiv.

[9]  Alexander L. Gaunt,et al.  Pushing the frontiers of density functionals by solving the fractional electron problem , 2021, Science.

[10]  Stephan Gunnemann,et al.  Graph Posterior Network: Bayesian Predictive Uncertainty for Node Classification , 2021, NeurIPS.

[11]  Stephan Günnemann,et al.  Ab-Initio Potential Energy Surfaces by Pairing GNNs with Neural Wave Functions , 2021, ICLR.

[12]  Connor W. Coley,et al.  Evidential Deep Learning for Guided Molecular Property Prediction and Discovery , 2021, ACS central science.

[13]  Xiao Xiang Zhu,et al.  A survey of uncertainty in deep neural networks , 2021, Artificial Intelligence Review.

[14]  Kalvik Jakkala,et al.  Deep Gaussian Processes: A Survey , 2021, ArXiv.

[15]  Florian Becker,et al.  GemNet: Universal Directional Graph Neural Networks for Molecules , 2021, NeurIPS.

[16]  Bertrand Charpentier,et al.  Natural Posterior Network: Deep Bayesian Uncertainty for Exponential Family Distributions , 2021, 2105.04471.

[17]  Klaus-Robert Müller,et al.  SpookyNet: Learning force fields with electronic degrees of freedom and nonlocal effects , 2021, Nature Communications.

[18]  Carl E. Rasmussen,et al.  The Promises and Pitfalls of Deep Kernel Learning , 2021, UAI.

[19]  Joost R. van Amersfoort,et al.  On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty , 2021, 2102.11409.

[20]  Michael Gastegger,et al.  Equivariant message passing for the prediction of tensorial properties and molecular spectra , 2021, ICML.

[21]  Jonathan P. Mailoa,et al.  E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials , 2021, Nature Communications.

[22]  A. Micheli,et al.  Graph Mixture Density Networks , 2020, ICML.

[23]  Johannes T. Margraf,et al.  Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Molecules , 2020, ArXiv.

[24]  Brooks Paige,et al.  Bayesian Graph Neural Networks for Molecular Property Prediction , 2020, 2012.02089.

[25]  Stephan Günnemann,et al.  Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable? , 2020, ICML.

[26]  W. Hsu,et al.  Towards Maximizing the Representation Gap between In-Domain \& Out-of-Distribution Examples , 2020, NeurIPS.

[27]  Zhihui Li,et al.  A Survey of Deep Active Learning , 2020, ACM Comput. Surv..

[28]  Frederick R. Manby,et al.  OrbNet: Deep Learning for Quantum Chemistry Using Symmetry-Adapted Atomic-Orbital Features , 2020, The Journal of chemical physics.

[29]  A. Tkatchenko,et al.  QM7-X: A comprehensive dataset of quantum-mechanical properties spanning the chemical space of small organic molecules , 2020, 2006.15139.

[30]  Jasper Snoek,et al.  Hyperparameter Ensembles for Robustness and Uncertainty Quantification , 2020, NeurIPS.

[31]  Tengyu Ma,et al.  Individual Calibration with Randomized Forecasting , 2020, ICML.

[32]  Dustin Tran,et al.  Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness , 2020, NeurIPS.

[33]  Stephan Günnemann,et al.  Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts , 2020, NeurIPS.

[34]  Murat Sensoy,et al.  Uncertainty-Aware Deep Classifiers Using Generative Models , 2020, AAAI.

[35]  Byron Boots,et al.  Intra Order-preserving Functions for Calibration of Multi-Class Neural Networks , 2020, NeurIPS.

[36]  Stephan Günnemann,et al.  Directional Message Passing for Molecular Graphs , 2020, ICLR.

[37]  Dustin Tran,et al.  BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning , 2020, ICLR.

[38]  Thomas Brox,et al.  Parting with Illusions about Deep Active Learning , 2019, ArXiv.

[39]  Stephan Günnemann,et al.  Uncertainty on Asynchronous Time Event Prediction , 2019, NeurIPS.

[40]  Lars A. Bratholm,et al.  FCHL revisited: Faster and more accurate quantum machine learning. , 2019, The Journal of chemical physics.

[41]  Federico Tombari,et al.  Sampling-Free Epistemic Uncertainty Estimation Using Approximated Variance Propagation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[42]  Sebastian Nowozin,et al.  Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift , 2019, NeurIPS.

[43]  Tom Diethe,et al.  Distribution Calibration for Regression , 2019, ICML.

[44]  Simon L. Batzner,et al.  On-the-fly active learning of interpretable Bayesian force fields for atomistic rare events , 2019, npj Computational Materials.

[45]  Markus Meuwly,et al.  PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments, and Partial Charges. , 2019, Journal of chemical theory and computation.

[46]  Andrew Gordon Wilson,et al.  A Simple Baseline for Bayesian Uncertainty in Deep Learning , 2019, NeurIPS.

[47]  Boris Flach,et al.  Feed-forward Propagation in Probabilistic Neural Networks with Categorical and Max Layers , 2018, ICLR.

[48]  Stefano Ermon,et al.  Accurate Uncertainties for Deep Learning Using Calibrated Regression , 2018, ICML.

[49]  S. Roth,et al.  Lightweight Probabilistic Deep Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[50]  Li Li,et al.  Tensor Field Networks: Rotation- and Translation-Equivariant Neural Networks for 3D Point Clouds , 2018, ArXiv.

[51]  David Barber,et al.  A Scalable Laplace Approximation for Neural Networks , 2018, ICLR.

[52]  K-R Müller,et al.  SchNet - A deep learning architecture for molecules and materials. , 2017, The Journal of chemical physics.

[53]  Charles Blundell,et al.  Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles , 2016, NIPS.

[54]  Klaus-Robert Müller,et al.  Machine learning of accurate energy-conserving molecular force fields , 2016, Science Advances.

[55]  Dit-Yan Yeung,et al.  Natural-Parameter Networks: A Class of Probabilistic Neural Networks , 2016, NIPS.

[56]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[57]  Ryan P. Adams,et al.  Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks , 2015, ICML.

[58]  James Hensman,et al.  Scalable Variational Gaussian Process Classification , 2014, AISTATS.

[59]  R. Kondor,et al.  On representing chemical environments , 2012, 1209.3140.

[60]  Stephen J. Roberts,et al.  Dynamic Bayesian Combination of Multiple Imperfect Classifiers , 2012, Decision Making and Imperfection.

[61]  Hyun-Chul Kim,et al.  Bayesian Classifier Combination , 2012, AISTATS.

[62]  Klaus-Robert Müller,et al.  Finding Density Functionals with Machine Learning , 2011, Physical review letters.

[63]  J. Behler Atom-centered symmetry functions for constructing high-dimensional neural network potentials. , 2011, The Journal of chemical physics.

[64]  Jan H. Jensen Molecular Modeling Basics , 2010 .

[65]  Michalis K. Titsias,et al.  Variational Learning of Inducing Variables in Sparse Gaussian Processes , 2009, AISTATS.

[66]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[67]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[68]  T. Halgren Merck molecular force field. I. Basis, form, scope, parameterization, and performance of MMFF94 , 1996, J. Comput. Chem..

[69]  S. Ji,et al.  Spherical Message Passing for 3D Molecular Graphs , 2022, ICLR.

[70]  Uncertainty Estimation Using a Single Deep Deterministic Neural Network-ML Reproducibility Challenge 2020 , 2021 .

[71]  Thomas F. Miller,et al.  UNiTE: Unitary N-body Tensor Equivariant Network with Applications to Quantum Chemistry , 2021, ArXiv.

[72]  Dennis Ulmer A Survey on Evidential Deep Learning For Single-Pass Uncertainty Estimation , 2021, ArXiv.

[73]  Feng Chen,et al.  Multifaceted Uncertainty Estimation for Label-Efficient Deep Learning , 2020, NeurIPS.

[74]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[75]  T. Ebisuzaki,et al.  Molecular Dynamics Machine: Special-Purpose Computer for Molecular Dynamics Simulations , 1999 .

[76]  J. Crabbe,et al.  Molecular modelling: Principles and applications , 1997 .