Wigner kernels: body-ordered equivariant machine learning without a basis

Machine-learning models based on a point-cloud representation of a physical object are ubiquitous in scientific applications and particularly well-suited to the atomic-scale description of molecules and materials. Among the many different approaches that have been pursued, the description of local atomic environments in terms of their neighbor densities has been used widely and very succesfully. We propose a novel density-based method which involves computing ``Wigner kernels''. These are fully equivariant and body-ordered kernels that can be computed iteratively with a cost that is independent of the radial-chemical basis and grows only linearly with the maximum body-order considered. This is in marked contrast to feature-space models, which comprise an exponentially-growing number of terms with increasing order of correlations. We present several examples of the accuracy of models based on Wigner kernels in chemical applications, for both scalar and tensorial targets, reaching state-of-the-art accuracy on the popular QM9 benchmark dataset, and we discuss the broader relevance of these ideas to equivariant geometric machine-learning.

[1]  Simon L. Batzner,et al.  Learning local equivariant representations for large-scale atomistic dynamics , 2022, Nature Communications.

[2]  Bruno Loureiro,et al.  Learning curves for the multi-class teacher–student perceptron , 2022, Mach. Learn. Sci. Technol..

[3]  James P. Darby,et al.  Tensor-reduced atomic density representations , 2022, Physical review letters.

[4]  D. Manolopoulos,et al.  A smooth basis for atomistic machine learning , 2022, Journal of Chemical Physics.

[5]  M. Ceriotti,et al.  Beyond potentials: Integrated machine learning models for materials , 2022, MRS Bulletin.

[6]  Gábor Csányi,et al.  MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields , 2022, NeurIPS.

[7]  Simon L. Batzner,et al.  The Design Space of E(3)-Equivariant Atom-Centered Interatomic Potentials , 2022, ArXiv.

[8]  Jigyasa Nigam,et al.  Unified theory of atom-centered representations and message-passing machine-learning schemes. , 2022, The Journal of chemical physics.

[9]  Yaolong Zhang,et al.  REANN: A PyTorch-based end-to-end multi-functional deep neural network package for molecular, reactive, and periodic systems. , 2021, The Journal of chemical physics.

[10]  Michele Ceriotti,et al.  Equivariant representations for molecular Hamiltonians and N-center atomic-scale properties. , 2021, The Journal of chemical physics.

[11]  P. Battaglia,et al.  Simple GNN Regularisation for 3D Molecular Property Prediction&Beyond , 2021, 2106.07971.

[12]  Jonathan P. Mailoa,et al.  E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials , 2021, Nature Communications.

[13]  Cas van der Oord,et al.  Atomic cluster expansion: Completeness, efficiency and stability , 2019, J. Comput. Phys..

[14]  G. D. Fabritiis,et al.  TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentials , 2022, ICLR.

[15]  Gábor Csányi,et al.  Local invertibility and sensitivity of atomic structure-feature mappings , 2021, Open research Europe.

[16]  M. Ceriotti,et al.  Introduction: Machine Learning at the Atomic Scale. , 2021, Chemical reviews.

[17]  Volker L. Deringer,et al.  Gaussian Process Regression for Materials and Molecules , 2021, Chemical reviews.

[18]  Michele Ceriotti,et al.  Optimal radial basis for density-based atomic representations , 2021, The Journal of chemical physics.

[19]  Joan Bruna,et al.  Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges , 2021, ArXiv.

[20]  Michael Gastegger,et al.  Equivariant message passing for the prediction of tensorial properties and molecular spectra , 2021, ICML.

[21]  Michael J. Willatt,et al.  Efficient implementation of atom-density representations. , 2021, The Journal of chemical physics.

[22]  Gábor Csányi,et al.  Physics-Inspired Structural Representations for Molecules and Materials. , 2021, Chemical reviews.

[23]  Fei Liu,et al.  Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Mohammed Bennamoun,et al.  Deep Learning for 3D Point Clouds: A Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Johannes T. Margraf,et al.  Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Molecules , 2020, ArXiv.

[26]  Johannes Kästner,et al.  Gaussian Moments as Physically Inspired Molecular Descriptors for Accurate and Scalable Machine Learning Potentials. , 2020, Journal of chemical theory and computation.

[27]  Jigyasa Nigam,et al.  Recursive evaluation and iterative contraction of N-body equivariant features. , 2020, The Journal of chemical physics.

[28]  M. Ceriotti,et al.  Predicting molecular dipole moments by combining atomic partial charges and atomic dipoles. , 2020, The Journal of chemical physics.

[29]  Christoph Ortner,et al.  Incompleteness of Atomic Structure Representations. , 2020, Physical review letters.

[30]  Cas van der Oord,et al.  Regularised atomic body-ordered permutation-invariant polynomials for the construction of interatomic potentials , 2019, Mach. Learn. Sci. Technol..

[31]  M. Bronstein,et al.  Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning , 2019, Nature Methods.

[32]  Risi Kondor,et al.  Cormorant: Covariant Molecular Neural Networks , 2019, NeurIPS.

[33]  Naftali Tishby,et al.  Machine learning and the physical sciences , 2019, Reviews of Modern Physics.

[34]  Markus Meuwly,et al.  PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments, and Partial Charges. , 2019, Journal of chemical theory and computation.

[35]  Ralf Drautz,et al.  Atomic cluster expansion for accurate and transferable interatomic potentials , 2019, Physical Review B.

[36]  Matthias Scheffler,et al.  Two-to-three dimensional transition in neutral gold clusters: The crucial role of van der Waals interactions and temperature , 2018, Physical Review Materials.

[37]  Fuxin Li,et al.  PointConv: Deep Convolutional Networks on 3D Point Clouds , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Michele Ceriotti,et al.  Atom-density representations for machine learning. , 2018, The Journal of chemical physics.

[39]  Andreas Geiger,et al.  SphereNet: Learning Spherical Representations for Detection and Classification in Omnidirectional Images , 2018, ECCV.

[40]  Michele Ceriotti,et al.  A Data-Driven Construction of the Periodic Table of the Elements , 2018, 1807.00236.

[41]  Li Li,et al.  Tensor Field Networks: Rotation- and Translation-Equivariant Neural Networks for 3D Point Clouds , 2018, ArXiv.

[42]  Nicola Gaston,et al.  Building machine learning force fields for nanoclusters. , 2018, The Journal of chemical physics.

[43]  Aldo Glielmo,et al.  Efficient nonparametric n -body force fields from machine learning , 2018, 1801.04823.

[44]  Anders S. Christensen,et al.  Alchemical and structural distribution based representation for universal quantum machine learning. , 2017, The Journal of chemical physics.

[45]  K-R Müller,et al.  SchNet - A deep learning architecture for molecules and materials. , 2017, The Journal of chemical physics.

[46]  Andrea Grisafi,et al.  Symmetry-Adapted Machine Learning for Tensorial Properties of Atomistic Systems. , 2017, Physical review letters.

[47]  Noam Bernstein,et al.  Machine learning unifies the modeling of materials and molecules , 2017, Science Advances.

[48]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[49]  Klaus-Robert Müller,et al.  Machine learning of accurate energy-conserving molecular force fields , 2016, Science Advances.

[50]  Peter Sollich,et al.  Accurate interatomic force fields via machine learning with covariant kernels , 2016, 1611.03877.

[51]  Li Li,et al.  Bypassing the Kohn-Sham equations with machine learning , 2016, Nature Communications.

[52]  O. A. von Lilienfeld,et al.  Communication: Understanding molecular representations in machine learning: The role of uniqueness and target similarity. , 2016, The Journal of chemical physics.

[53]  Gábor Csányi,et al.  Comparing molecules and solids across structural and alchemical space. , 2015, Physical chemistry chemical physics : PCCP.

[54]  Alexander V. Shapeev,et al.  Moment Tensor Potentials: A Class of Systematically Improvable Interatomic Potentials , 2015, Multiscale Model. Simul..

[55]  Andreas W Götz,et al.  On the representation of many-body interactions in water. , 2015, The Journal of chemical physics.

[56]  O. A. von Lilienfeld,et al.  Transferable Atomic Multipole Machine Learning Models for Small Organic Molecules. , 2015, Journal of chemical theory and computation.

[57]  Jens K Nørskov,et al.  Investigation of Catalytic Finite-Size-Effects of Platinum Metal Clusters. , 2013, The journal of physical chemistry letters.

[58]  R. Kondor,et al.  On representing chemical environments , 2012, 1209.3140.

[59]  K. Müller,et al.  Fast and accurate modeling of molecular atomization energies with machine learning. , 2011, Physical review letters.

[60]  R. Kondor,et al.  Gaussian approximation potentials: the accuracy of quantum mechanics, without the electrons. , 2009, Physical review letters.

[61]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[62]  Michele Parrinello,et al.  Generalized neural-network representation of high-dimensional potential-energy surfaces. , 2007, Physical review letters.

[63]  W. Kohn,et al.  Nearsightedness of electronic matter. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[64]  Cleve B. Moler,et al.  Nineteen Dubious Ways to Compute the Exponential of a Matrix, Twenty-Five Years Later , 1978, SIAM Rev..

[65]  Stefan Gumhold,et al.  Feature Extraction From Point Clouds , 2001, IMR.

[66]  Aoki,et al.  Bond-order potentials: Theory and implementation. , 1996, Physical review. B, Condensed matter.

[67]  F. Ducastelle,et al.  Generalized cluster description of multicomponent systems , 1984 .

[68]  M. Finnis,et al.  A simple empirical N-body potential for transition metals , 1984 .