Operator Autoencoders: Learning Physical Operations on Encoded Molecular Graphs

Molecular dynamics simulations produce data with complex nonlinear dynamics. If the timestep behavior of such a dynamic system can be represented by a linear operator, future states can be inferred directly without expensive simulations. The use of an autoencoder in combination with a physical timestep operator allows both the relevant structural characteristics of the molecular graphs and the underlying physics of the system to be isolated during the training process. In this work, we develop a pipeline for establishing graph-structured representations of time-series volumetric data from molecular dynamics simulations. We then train an autoencoder to find nonlinear mappings to a latent space where future timesteps can be predicted through application of a linear operator trained in tandem with the autoencoder. Increasing the dimensionality of the autoencoder output is shown to improve the accuracy of the physical timestep operator.

[1]  Greg Mori,et al.  Graph Generation with Variational Recurrent Neural Network , 2019, ArXiv.

[2]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[3]  Clarence W. Rowley,et al.  Linearly-Recurrent Autoencoder Networks for Learning Dynamics , 2017, SIAM J. Appl. Dyn. Syst..

[4]  D. Jiao,et al.  Lennard-Jones interatomic potentials for the allotropes of carbon. , 2018, 1805.10614.

[5]  Steven L. Brunton,et al.  Deep learning for universal linear embeddings of nonlinear dynamics , 2017, Nature Communications.

[6]  Frank Noé,et al.  Author Correction: VAMPnets for deep learning of molecular kinetics , 2018, Nature Communications.

[7]  Hamid R. Rabiee,et al.  Deep Graph Generators: A Survey , 2020, IEEE Access.

[8]  Regina Barzilay,et al.  Hierarchical Generation of Molecular Graphs using Structural Motifs , 2020, ICML.

[9]  Jean Ponce,et al.  Finding Matches in a Haystack: A Max-Pooling Strategy for Graph Matching in the Presence of Outliers , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Yang Yu,et al.  Unsupervised Representation Learning with Deep Convolutional Neural Network for Remote Sensing Images , 2017, ICIG.

[11]  Steve Plimpton,et al.  Fast parallel algorithms for short-range molecular dynamics , 1993 .

[12]  Nikos Komodakis,et al.  GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders , 2018, ICANN.

[13]  Hiroshi Kajino,et al.  Molecular Hypergraph Grammar with its Application to Molecular Optimization , 2018, ICML.

[14]  Nils M. Kriege,et al.  Deep Graph Matching Consensus , 2020, ICLR.

[15]  B. O. Koopman,et al.  Hamiltonian Systems and Transformation in Hilbert Space. , 1931, Proceedings of the National Academy of Sciences of the United States of America.

[16]  Zhi Chen,et al.  Adversarial Feature Matching for Text Generation , 2017, ICML.

[17]  Igor Mezi'c,et al.  Geometry of the ergodic quotient reveals coherent structures in flows , 2012, 1204.2050.

[18]  Andrzej Banaszuk,et al.  Comparison of systems with complex behavior , 2004 .

[19]  Hirokazu Kameoka,et al.  Sequence-to-Sequence Voice Conversion with Similarity Metric Learned Using Generative Adversarial Networks , 2017, INTERSPEECH.

[20]  Steven L. Brunton,et al.  DeepGreen: deep learning of Green’s functions for nonlinear boundary value problems , 2020, Scientific Reports.

[21]  Cao Xiao,et al.  Constrained Generation of Semantically Valid Graphs via Regularizing Variational Autoencoders , 2018, NeurIPS.

[22]  Matt J. Kusner,et al.  A Model to Search for Synthesizable Molecules , 2019, NeurIPS.

[23]  Regina Barzilay,et al.  Junction Tree Variational Autoencoder for Molecular Graph Generation , 2018, ICML.

[24]  Zhiyuan Liu,et al.  Graph Neural Networks: A Review of Methods and Applications , 2018, AI Open.

[25]  I. Mezić,et al.  Analysis of Fluid Flows via Spectral Properties of the Koopman Operator , 2013 .

[26]  Hong Cheng,et al.  Dirichlet Graph Variational Autoencoder , 2020, NeurIPS.

[27]  Soumya Kundu,et al.  Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systems , 2017, 2019 American Control Conference (ACC).

[28]  William Yang Wang,et al.  Riemannian Normalizing Flow on Variational Wasserstein Autoencoder for Text Modeling , 2019, NAACL.

[29]  Wenwu Zhu,et al.  Deep Learning on Graphs: A Survey , 2018, IEEE Transactions on Knowledge and Data Engineering.

[30]  Honglak Lee,et al.  Attribute2Image: Conditional Image Generation from Visual Attributes , 2015, ECCV.

[31]  Edwin R. Hancock,et al.  Learning for Graph Matching and Related Combinatorial Optimization Problems , 2020, IJCAI.

[32]  Regina Barzilay,et al.  Learning Multimodal Graph-to-Graph Translation for Molecular Optimization , 2018, ICLR.

[33]  Niloy Ganguly,et al.  NeVAE: A Deep Generative Model for Molecular Graphs , 2018, AAAI.

[34]  Sang-Yeon Hwang,et al.  Scaffold-based molecular design using graph generative model , 2019, ArXiv.

[35]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[36]  Kathryn B. Laskey,et al.  Stochastic blockmodels: First steps , 1983 .

[37]  Alan Aspuru-Guzik,et al.  Graph Deconvolutional Generation , 2020, ArXiv.

[38]  Naoya Takeishi,et al.  Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition , 2017, NIPS.

[39]  Ryan A. Rossi,et al.  Attention Models in Graphs: A Survey , 2018 .

[40]  Wenhan Shi,et al.  Conditional Structure Generation through Graph Variational Generative Adversarial Nets , 2019, NeurIPS.

[41]  Yoshua Bengio,et al.  DEFactor: Differentiable Edge Factorization-based Probabilistic Graph Generation , 2018, ArXiv.

[42]  Aviral Kumar,et al.  Graph Normalizing Flows , 2019, NeurIPS.

[43]  I. Mezić Spectral Properties of Dynamical Systems, Model Reduction and Decompositions , 2005 .

[44]  Zhihai He,et al.  A Comprehensive Survey on Geometric Deep Learning , 2020, IEEE Access.

[45]  P. Erdos,et al.  On the evolution of random graphs , 1984 .

[46]  Philip S. Yu,et al.  Adversarial Attack and Defense on Graph Data: A Survey , 2018 .

[47]  Pushmeet Kohli,et al.  Graph Matching Networks for Learning the Similarity of Graph Structured Objects , 2019, ICML.

[48]  Qi Liu,et al.  Constrained Graph Variational Autoencoders for Molecule Design , 2018, NeurIPS.

[49]  Philip S. Yu,et al.  Deep graph similarity learning: a survey , 2019, Data Mining and Knowledge Discovery.

[50]  Ioannis G Kevrekidis,et al.  Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator. , 2017, Chaos.

[51]  Stefano Ermon,et al.  Graphite: Iterative Generative Modeling of Graphs , 2018, ICML.

[52]  Christos Faloutsos,et al.  Kronecker Graphs: An Approach to Modeling Networks , 2008, J. Mach. Learn. Res..

[53]  Hao Wu,et al.  VAMPnets for deep learning of molecular kinetics , 2017, Nature Communications.

[54]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[55]  Frank Noé,et al.  Time-lagged autoencoders: Deep learning of slow collective variables for molecular kinetics , 2017, The Journal of chemical physics.

[56]  Yang Gao,et al.  Voice Impersonation Using Generative Adversarial Networks , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).