Spherical Message Passing for 3D Molecular Graphs

We consider representation learning of 3D molecular graphs in which each atom is associated with a spatial position in 3D. This is an under-explored area of research, and a principled message passing framework is currently lacking. In this work, we conduct analyses in the spherical coordinate system (SCS) for the complete identification of 3D graph structures. Based on such observations, we propose the spherical message passing (SMP) as a novel and powerful scheme for 3D molecular learning. SMP dramatically reduces training complexity, enabling it to perform efficiently on large-scale molecules. In addition, SMP is capable of distinguishing almost all molecular structures, and the uncovered cases may not exist in practice. Based on meaningful physically-based representations of 3D information, we further propose the SphereNet for 3D molecular learning. Experimental results demonstrate that the use of meaningful 3D information in SphereNet leads to significant performance improvements in prediction tasks. Our results also demonstrate the advantages of SphereNet in terms of capability, efficiency, and scalability. Our code is publicly available as part of the DIG library (https://github.com/divelab/DIG).

[1]  P. Battaglia,et al.  Simple GNN Regularisation for 3D Molecular Property Prediction&Beyond , 2021, 2106.07971.

[2]  Jonathan P. Mailoa,et al.  E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials , 2021, Nature Communications.

[3]  Shuiwang Ji,et al.  Advanced Graph and Sequence Neural Networks for Molecular Property Prediction and Drug Discovery. , 2020, Bioinformatics.

[4]  Simon Axelrod,et al.  GEOM: Energy-annotated molecular conformations for property prediction and molecular generation , 2020, ArXiv.

[5]  Shuiwang Ji,et al.  Graph U-Nets , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Florian Becker,et al.  GemNet: Universal Directional Graph Neural Networks for Molecules , 2021, NeurIPS.

[7]  Joan Bruna,et al.  Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges , 2021, ArXiv.

[8]  Shuiwang Ji,et al.  DIG: A Turnkey Library for Diving into Graph Deep Learning Research , 2021, J. Mach. Learn. Res..

[9]  J. Leskovec,et al.  ForceNet: A Graph Neural Network for Large-Scale Quantum Calculations , 2021, ArXiv.

[10]  Michael Gastegger,et al.  Equivariant message passing for the prediction of tensorial properties and molecular spectra , 2021, ICML.

[11]  Shuiwang Ji,et al.  Topology-Aware Graph Pooling Networks , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Max Welling,et al.  Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs , 2020, ICLR.

[13]  Mohammed Bennamoun,et al.  Deep Learning for 3D Point Clouds: A Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Johannes T. Margraf,et al.  Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Molecules , 2020, ArXiv.

[15]  Weihua Hu,et al.  The Open Catalyst 2020 (OC20) Dataset and Community Challenges , 2020, ACS Catalysis.

[16]  Frederick R. Manby,et al.  OrbNet: Deep Learning for Quantum Chemistry Using Symmetry-Adapted Atomic-Orbital Features , 2020, The Journal of chemical physics.

[17]  Shuiwang Ji,et al.  Deep Learning of High-Order Interactions for Protein Interface Prediction , 2020, KDD.

[18]  Andreas Loukas,et al.  Building powerful and equivariant graph neural networks with message-passing , 2020, ArXiv.

[19]  Andreas Loukas,et al.  Building powerful and equivariant graph neural networks with structural message-passing , 2020, NeurIPS.

[20]  Fabian B. Fuchs,et al.  SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks , 2020, NeurIPS.

[21]  Stephan Günnemann,et al.  Directional Message Passing for Molecular Graphs , 2020, ICLR.

[22]  Jure Leskovec,et al.  Learning to Simulate Complex Physics with Graph Networks , 2020, ICML.

[23]  José Miguel Hernández-Lobato,et al.  Reinforcement Learning for Molecular Design Guided by Quantum Mechanics , 2020, ICML.

[24]  Stefanie Jegelka,et al.  Generalization and Representational Limits of Graph Neural Networks , 2020, ICML.

[25]  Qi Liu,et al.  Molecular Property Prediction: A Multilevel Quantum Interactions Modeling Perspective , 2019, AAAI.

[26]  Risi Kondor,et al.  Cormorant: Covariant Molecular Neural Networks , 2019, NeurIPS.

[27]  Lei Wang,et al.  Appendix for : Graph Attention Convolution for Point Cloud Semantic Segmentation , 2019 .

[28]  Chao Chen,et al.  ClusterNet: Deep Hierarchical Cluster Network With Rigorously Rotation-Invariant Representation for Point Cloud Analysis , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Yaron Lipman,et al.  Provably Powerful Graph Networks , 2019, NeurIPS.

[30]  Mohamed Boussaha,et al.  Point Cloud Oversegmentation With Graph-Structured Deep Metric Learning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Markus Meuwly,et al.  PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments, and Partial Charges. , 2019, Journal of chemical theory and computation.

[32]  Max Welling,et al.  Gauge Equivariant Convolutional Networks and the Icosahedral CNN 1 , 2019 .

[33]  Nathanael Perraudin,et al.  DeepSphere: Efficient spherical Convolutional Neural Network with HEALPix sampling for cosmological applications , 2018, Astron. Comput..

[34]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[35]  Raphael J. L. Townshend,et al.  End-to-End Learning on 3D Protein Structure for Interface Prediction , 2018, NeurIPS.

[36]  Yingyu Liang,et al.  N-Gram Graph: Simple Unsupervised Representation for Graphs, with Applications to Molecules , 2018, NeurIPS.

[37]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[38]  Zhengyang Wang,et al.  Large-Scale Learnable Graph Convolutional Networks , 2018, KDD.

[39]  Razvan Pascanu,et al.  Relational inductive biases, deep learning, and graph networks , 2018, ArXiv.

[40]  Yixin Chen,et al.  An End-to-End Deep Learning Architecture for Graph Classification , 2018, AAAI.

[41]  K. Müller,et al.  Towards exact molecular dynamics simulations with machine-learned force fields , 2018, Nature Communications.

[42]  Li Li,et al.  Tensor Field Networks: Rotation- and Translation-Equivariant Neural Networks for 3D Point Clouds , 2018, ArXiv.

[43]  Dong Tian,et al.  Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[44]  Martin Simonovsky,et al.  Large-Scale Point Cloud Semantic Segmentation with Superpoint Graphs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[45]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[46]  Jeffrey C Grossman,et al.  Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties. , 2017, Physical review letters.

[47]  Alex Fout,et al.  Protein Interface Prediction using Graph Convolutional Networks , 2017, NIPS.

[48]  Klaus-Robert Müller,et al.  SchNet: A continuous-filter convolutional neural network for modeling quantum interactions , 2017, NIPS.

[49]  Nikos Komodakis,et al.  Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[51]  Vijay S. Pande,et al.  MoleculeNet: a benchmark for molecular machine learning , 2017, Chemical science.

[52]  Klaus-Robert Müller,et al.  Machine learning of accurate energy-conserving molecular force fields , 2016, Science Advances.

[53]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[54]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[55]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[56]  Pavlo O. Dral,et al.  Quantum chemistry structures and properties of 134 kilo molecules , 2014, Scientific Data.

[57]  Kurt Mehlhorn,et al.  Weisfeiler-Lehman Graph Kernels , 2011, J. Mach. Learn. Res..

[58]  F. Scarselli,et al.  A new model for learning in graph domains , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[59]  E. Villaseñor.  Introduction to Quantum Mechanics , 2008, Nature.