Symmetry-Aware Actor-Critic for 3D Molecular Design

Automating molecular design using deep reinforcement learning (RL) has the potential to greatly accelerate the search for novel materials. Despite recent progress on leveraging graph representations to design molecules, such methods are fundamentally limited by the lack of three-dimensional (3D) information. In light of this, we propose a novel actor-critic architecture for 3D molecular design that can generate molecular structures unattainable with previous approaches. This is achieved by exploiting the symmetries of the design process through a rotationally covariant state-action representation based on a spherical harmonics series expansion. We demonstrate the benefits of our approach on several 3D molecular design tasks, where we find that building in such symmetries significantly improves generalization and the quality of generated molecules.

[1]  V. Lebedev.  Values of the nodes and weights of ninth to seventeenth order gauss-markov quadrature formulae invariant under the octahedron group with inversion☆ , 1975 .

[2]  Olexandr Isayev,et al.  Deep reinforcement learning for de novo drug design , 2017, Science Advances.

[3]  Li Li,et al.  Optimization of Molecules via Deep Reinforcement Learning , 2018, Scientific Reports.

[4]  G. Schneider,et al.  Rethinking drug design in the artificial intelligence era , 2019, Nature Reviews Drug Discovery.

[5]  Alán Aspuru-Guzik,et al.  Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models , 2017, ArXiv.

[6]  Razvan Pascanu,et al.  Learning Deep Generative Models of Graphs , 2018, ICLR 2018.

[7]  Risi Kondor,et al.  On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups , 2018, ICML.

[8]  Jos'e Miguel Hern'andez-Lobato,et al.  Reinforcement Learning for Molecular Design Guided by Quantum Mechanics , 2020, ICML.

[9]  H. Bateman,et al.  Higher Transcendental Functions [Volumes I-III] , 1953 .

[10]  Nicola De Cao,et al.  MolGAN: An implicit generative model for small molecular graphs , 2018, ArXiv.

[11]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[12]  Yibo Li,et al.  Multi-objective de novo drug design with conditional graph generative model , 2018, Journal of Cheminformatics.

[13]  Mohamed Ahmed,et al.  Exploring Deep Recurrent Models with Reinforcement Learning for Molecule Design , 2018, ICLR.

[14]  Woo Youn Kim,et al.  Universal Structure Conversion Method for Organic Molecules: From Atomic Connectivity to Three‐Dimensional Geometry , 2015 .

[15]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[16]  Regina Barzilay,et al.  Predicting Organic Reaction Outcomes with Weisfeiler-Lehman Network , 2017, NIPS.

[17]  Alán Aspuru-Guzik,et al.  Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules , 2016, ACS central science.

[18]  Qi Liu,et al.  Constrained Graph Variational Autoencoders for Molecule Design , 2018, NeurIPS.

[19]  Markus Reiher,et al.  Semiempirical molecular orbital models based on the neglect of diatomic differential overlap approximation , 2018, International Journal of Quantum Chemistry.

[20]  Risi Kondor,et al.  Cormorant: Covariant Molecular Neural Networks , 2019, NeurIPS.

[21]  Michael Gastegger,et al.  Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules , 2019, NeurIPS.

[22]  Alexander J. Smola,et al.  Deep Sets , 2017, 1703.06114.

[23]  Li Li,et al.  Tensor Field Networks: Rotation- and Translation-Equivariant Neural Networks for 3D Point Clouds , 2018, ArXiv.

[24]  S. Srihari.  Mixture Density Networks , 1994 .

[25]  Thomas Blaschke,et al.  Molecular de-novo design through deep reinforcement learning , 2017, Journal of Cheminformatics.

[26]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[27]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[28]  J. Stewart.  Optimization of parameters for semiempirical methods V: Modification of NDDO approximations and application to 70 elements , 2007, Journal of molecular modeling.

[29]  V. I. Lebedev,et al.  Spherical quadrature formulas exact to orders 25–29 , 1977 .

[30]  Thierry Kogej,et al.  Generating Focused Molecule Libraries for Drug Discovery with Recurrent Neural Networks , 2017, ACS central science.

[31]  Bjørk Hammer,et al.  Atomistic structure learning , 2019, The Journal of Chemical Physics.

[32]  Max Welling,et al.  3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data , 2018, NeurIPS.

[33]  Herke van Hoof,et al.  MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning , 2020, NeurIPS.

[34]  E. B. Andersen,et al.  Information Science and Statistics , 1986 .

[35]  Jure Leskovec,et al.  Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation , 2018, NeurIPS.

[36]  Michael Gastegger,et al.  Generating equilibrium molecules with deep neural networks , 2018, ArXiv.

[37]  Zhen Lin,et al.  Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network , 2018, NeurIPS.

[38]  Matt J. Kusner,et al.  A Model to Search for Synthesizable Molecules , 2019, NeurIPS.

[39]  Wes McKinney,et al.  Data Structures for Statistical Computing in Python , 2010, SciPy.

[40]  Markus Reiher,et al.  Comprehensive Analysis of the Neglect of Diatomic Differential Overlap Approximation. , 2018, Journal of chemical theory and computation.

[41]  Fabian B. Fuchs,et al.  SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks , 2020, NeurIPS.

[42]  Andrew Gordon Wilson,et al.  Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data , 2020, ICML.

[43]  Mario Geiger,et al.  Relevance of Rotationally Equivariant Convolutions for Predicting Molecular Properties , 2020, ArXiv.

[44]  Alán Aspuru-Guzik,et al.  Reinforced Adversarial Neural Computer for de Novo Molecular Design , 2018, J. Chem. Inf. Model..

[45]  Henrik Lund Mortensen,et al.  Structure prediction of surface reconstructions by deep reinforcement learning , 2020, Journal of physics. Condensed matter : an Institute of Physics journal.

[46]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[47]  Marwin H. S. Segler,et al.  GuacaMol: Benchmarking Models for De Novo Molecular Design , 2018, J. Chem. Inf. Model..

[48]  Matt J. Kusner,et al.  A Generative Model For Electron Paths , 2018, ICLR.

[49]  S. Rao Jammalamadaka,et al.  Harmonic analysis and distribution-free inference for spherical distributions , 2017, J. Multivar. Anal..