Crystal Structure Prediction by Joint Equivariant Diffusion

Crystal Structure Prediction (CSP) is crucial in various scientific disciplines. While CSP can be addressed by employing currently-prevailing generative models (e.g. diffusion models), this task encounters unique challenges owing to the symmetric geometry of crystal structures -- the invariance of translation, rotation, and periodicity. To incorporate the above symmetries, this paper proposes DiffCSP, a novel diffusion model to learn the structure distribution from stable crystals. To be specific, DiffCSP jointly generates the lattice and atom coordinates for each crystal by employing a periodic-E(3)-equivariant denoising model, to better model the crystal geometry. Notably, different from related equivariant generative approaches, DiffCSP leverages fractional coordinates other than Cartesian coordinates to represent crystals, remarkably promoting the diffusion and the generation process of atom positions. Extensive experiments verify that our DiffCSP significantly outperforms existing CSP methods, with a much lower computation cost in contrast to DFT-based methods. Moreover, the superiority of DiffCSP is also observed when it is extended for ab initio crystal generation.

[1]  Jian Peng,et al.  Antigen-Specific Antibody Design and Optimization with Diffusion-Based Generative Models for Protein Structures , 2022, bioRxiv.

[2]  T. Jaakkola,et al.  DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docking , 2022, ICLR.

[3]  Zhongkai Hao,et al.  Equivariant Energy-Guided SDE for Inverse Molecular Design , 2022, ICLR.

[4]  V. Cevher,et al.  DiGress: Discrete Denoising diffusion for graph generation , 2022, ICLR.

[5]  Shuiwang Ji,et al.  Periodic Graph Transformers for Crystal Material Property Prediction , 2022, NeurIPS.

[6]  Aaron C. Courville,et al.  Riemannian Diffusion Models , 2022, NeurIPS.

[7]  Chongxuan Li,et al.  EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equations , 2022, NeurIPS.

[8]  Zachary W. Ulissi,et al.  The Open Catalyst 2022 (OC22) Dataset and Challenges for Oxide Electrocatalysis , 2022, ACS Catalysis.

[9]  Brian L. Trippe,et al.  Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem , 2022, ICLR.

[10]  T. Jaakkola,et al.  Torsional Diffusion for Molecular Conformer Generation , 2022, NeurIPS.

[11]  Prafulla Dhariwal,et al.  Hierarchical Text-Conditional Image Generation with CLIP Latents , 2022, ArXiv.

[12]  Victor Garcia Satorras,et al.  Equivariant Diffusion for Molecule Generation in 3D , 2022, ICML.

[13]  W. Yin,et al.  Crystal structure prediction by combining graph network and optimization algorithm , 2022, Nature Communications.

[14]  S. Ermon,et al.  GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation , 2022, ICLR.

[15]  Valentin De Bortoli,et al.  Riemannian Score-Based Generative Modeling , 2022, NeurIPS.

[16]  B. Ommer,et al.  High-Resolution Image Synthesis with Latent Diffusion Models , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  T. Jaakkola,et al.  Crystal Diffusion Variational Autoencoder for Periodic Material Generation , 2021, ICLR.

[18]  Michael Gastegger,et al.  Inverse design of 3d molecular structures with conditional generative neural networks , 2021, Nature Communications.

[19]  Oriol Vinyals,et al.  Highly accurate protein structure prediction with AlphaFold , 2021, Nature.

[20]  Florian Becker,et al.  GemNet: Universal Directional Graph Neural Networks for Molecules , 2021, NeurIPS.

[21]  Jian Tang,et al.  Learning Gradient Fields for Molecular Conformation Generation , 2021, ICML.

[22]  Rongzhi Dong,et al.  Crystal structure prediction of materials with high symmetry using differential evolution , 2021, Journal of physics. Condensed matter : an Institute of Physics journal.

[23]  Max Welling,et al.  E(n) Equivariant Graph Neural Networks , 2021, ICML.

[24]  Prafulla Dhariwal,et al.  Improved Denoising Diffusion Probabilistic Models , 2021, ICML.

[25]  Didrik Nielsen,et al.  Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions , 2021, NeurIPS.

[26]  Johannes T. Margraf,et al.  Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Molecules , 2020, ArXiv.

[27]  Abhishek Kumar,et al.  Score-Based Generative Modeling through Stochastic Differential Equations , 2020, ICLR.

[28]  Weihua Hu,et al.  The Open Catalyst 2020 (OC20) Dataset and Community Challenges , 2020, ACS Catalysis.

[29]  Edirisuriya M Dilanga Siriwardane,et al.  Distance Matrix-Based Crystal Structure Prediction Using Evolutionary Algorithms. , 2020, The journal of physical chemistry. A.

[30]  Callum Court,et al.  3-D Inorganic Crystal Structure Generation and Property Prediction via Representation Learning , 2020, J. Chem. Inf. Model..

[31]  Edirisuriya M. D. Siriwardane,et al.  Contact map based crystal structure prediction using global optimization , 2020, CrystEngComm.

[32]  Pieter Abbeel,et al.  Denoising Diffusion Probabilistic Models , 2020, NeurIPS.

[33]  Fabian B. Fuchs,et al.  SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks , 2020, NeurIPS.

[34]  Stefano Ermon,et al.  Improved Techniques for Training Score-Based Generative Models , 2020, NeurIPS.

[35]  Tonio Buonassisi,et al.  An Invertible Crystallographic Representation for General Inverse Design of Inorganic Crystals with Targeted Properties , 2020, SSRN Electronic Journal.

[36]  Sungwon Kim,et al.  Generative Adversarial Networks for Crystal Structure Prediction , 2020, ACS central science.

[37]  C. Pickard AIRSS data for carbon at 10GPa and the C+N+H+O system at 1GPa , 2020 .

[38]  Alán Aspuru-Guzik,et al.  Inverse Design of Solid-State Materials via a Continuous Representation , 2019, Matter.

[39]  Yoshua Bengio,et al.  Data-Driven Approach to Encoding and Decoding 3-D Crystal Structures , 2019, ArXiv.

[40]  Aron Walsh,et al.  SMACT: Semiconducting Materials by Analogy and Chemical Theory , 2019, J. Open Source Softw..

[41]  Michael Gastegger,et al.  Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules , 2019, NeurIPS.

[42]  Nils E. R. Zimmermann,et al.  Local structure order parameters and site fingerprints for quantification of coordination environment and crystal structure similarity , 2019, RSC advances.

[43]  Artem R. Oganov,et al.  Structure prediction drives materials discovery , 2019, Nature Reviews Materials.

[44]  Chi Chen,et al.  Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals , 2018, Chemistry of Materials.

[45]  Nataliya Sokolovska,et al.  CrystalGAN: Learning to Discover Crystallographic Structures with Generative Adversarial Networks , 2018, AAAI Spring Symposium: Combining Machine Learning with Knowledge Engineering.

[46]  K. Butler,et al.  Machine learning for molecular and materials science , 2018, Nature.

[47]  Li Li,et al.  Tensor Field Networks: Rotation- and Translation-Equivariant Neural Networks for 3D Point Clouds , 2018, ArXiv.

[48]  Alexander V. Shapeev,et al.  Accelerating crystal structure prediction by machine-learning interatomic potentials with active learning , 2018, Physical Review B.

[49]  B. Hammer,et al.  On-the-Fly Machine Learning of Atomic Potential in Density Functional Theory Structure Optimization. , 2018, Physical review letters.

[50]  Takashi Miyake,et al.  Crystal structure prediction accelerated by Bayesian optimization , 2018 .

[51]  K-R Müller,et al.  SchNet - A deep learning architecture for molecules and materials. , 2017, The Journal of chemical physics.

[52]  Jeffrey C Grossman,et al.  Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties. , 2017, Physical review letters.

[53]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[54]  Hui Wang,et al.  Computer-Assisted Inverse Design of Inorganic Electrides , 2017 .

[55]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[56]  Logan T. Ward,et al.  A General-Purpose Machine Learning Framework for Predicting Properties of Inorganic Materials , 2016, 1606.09551.

[57]  Surya Ganguli,et al.  Deep Unsupervised Learning using Nonequilibrium Thermodynamics , 2015, ICML.

[58]  Gerhard Kurz,et al.  Efficient evaluation of the probability density function of a wrapped normal distribution , 2014, 2014 Sensor Data Fusion: Trends, Solutions, Applications (SDF).

[59]  Kristin A. Persson,et al.  Commentary: The Materials Project: A materials genome approach to accelerating materials innovation , 2013 .

[60]  David D. Cox,et al.  Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures , 2013, ICML.

[61]  Anubhav Jain,et al.  Python Materials Genomics (pymatgen): A robust, open-source python library for materials analysis , 2012 .

[62]  Thomas F. Jaramillo,et al.  New cubic perovskites for one- and two-photon water splitting using the computational materials repository , 2012 .

[63]  Thomas Olsen,et al.  Computational screening of perovskite metal oxides for optimal solar light capture , 2012 .

[64]  Chris J Pickard,et al.  Ab initio random structure searching , 2011, Journal of physics. Condensed matter : an Institute of Physics journal.

[65]  Jian Lv,et al.  Crystal structure prediction via particle-swarm optimization , 2010, 1008.3601.

[66]  Nikolaus Hansen,et al.  USPEX - Evolutionary crystal structure prediction , 2006, Comput. Phys. Commun..

[67]  Thomas Lengauer,et al.  Crystal structure prediction by data mining , 2002 .

[68]  Burke,et al.  Generalized Gradient Approximation Made Simple. , 1996, Physical review letters.

[69]  G. Kresse,et al.  Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set , 1996 .

[70]  Blöchl,et al.  Projector augmented-wave method. , 1994, Physical review. B, Condensed matter.

[71]  W. Kohn,et al.  Self-Consistent Equations Including Exchange and Correlation Effects , 1965 .

[72]  G. D. Fabritiis,et al.  TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentials , 2022, ICLR.

[73]  P D Adams,et al.  Numerically stable algorithms for the computation of reduced unit cells. , 2004, Acta crystallographica. Section A, Foundations of crystallography.

[74]  Gautam R. Desiraju,et al.  Cryptic crystallography , 2002, Nature materials.