Deep Representation Learning for Complex Free Energy Landscapes.

In this letter, we analyzed the inductive bias underlying complex free energy landscapes (FEL), and exploited it to train deep neural networks which yield reduced and clustered representation for the FEL. Our parametric method, called Information Distilling of Metastability (IDM), is end-to-end differentiable thus scalable to ultra-large dataset. IDM is able to perform clustering in the meantime of reducing the dimensions. Besides, as an unsupervised learning method, IDM differs from many existing dimensionality reduction and clustering methods in that it neither requires a cherry-picked distance metric nor the ground-true number of clusters defined a priori, and that it can be used to unroll and zoom-in the hierarchical FEL with respect to different timescales. Through multiple experiments, we show that IDM can achieve physically meaningful representations which partition the FEL into well-defined metastable states hence are amenable for downstream tasks such as mechanism analysis and kinetic modeling.

[1]  V. Pande,et al.  Chemical kinetics and mechanisms of complex systems: a perspective on recent theoretical advances. , 2014, Journal of the American Chemical Society.

[2]  Aaron C. Courville,et al.  Adversarially Learned Inference , 2016, ICLR.

[3]  Xu Ji,et al.  Invariant Information Clustering for Unsupervised Image Classification and Segmentation , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[4]  Frank Noé,et al.  Markov state models of biomolecular conformational dynamics. , 2014, Current opinion in structural biology.

[5]  Xu Ji,et al.  Invariant Information Distillation for Unsupervised Image Segmentation and Clustering , 2018, ArXiv.

[6]  Michele Parrinello,et al.  Simplifying the representation of complex free-energy landscapes using sketch-map , 2011, Proceedings of the National Academy of Sciences.

[7]  David Barber,et al.  Thinking Fast and Slow with Deep Learning and Tree Search , 2017, NIPS.

[8]  J. Hartigan.  Direct Clustering of a Data Matrix , 1972 .

[9]  R. Dror,et al.  How Fast-Folding Proteins Fold , 2011, Science.

[10]  Geoffrey E. Hinton,et al.  Self-organizing neural network that discovers surfaces in random-dot stereograms , 1992, Nature.

[11]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[12]  E. Olivieri,et al.  Large deviations and metastability , 2005 .

[13]  Gerhard Hummer,et al.  Position-dependent diffusion coefficients and free energies from Bayesian analysis of equilibrium and replica molecular dynamics simulations , 2005 .

[14]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[15]  Gerhard Stock,et al.  How complex is the dynamics of Peptide folding? , 2007, Physical review letters.

[16]  P. Wolynes,et al.  The energy landscapes and motions of proteins. , 1991, Science.

[17]  G. Hummer,et al.  Coarse master equations for peptide folding dynamics. , 2008, The journal of physical chemistry. B.

[18]  G. Torrie,et al.  Nonphysical sampling distributions in Monte Carlo free-energy estimation: Umbrella sampling , 1977 .

[19]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[20]  Sebastian Thrun,et al.  Lifelong Learning Algorithms , 1998, Learning to Learn.

[21]  K. Dill,et al.  Automatic discovery of metastable states for the construction of Markov models of macromolecular conformational dynamics. , 2007, The Journal of chemical physics.

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  Sebastian Ruder,et al.  An Overview of Multi-Task Learning in Deep Neural Networks , 2017, ArXiv.

[24]  P. Hänggi,et al.  Reaction-rate theory: fifty years after Kramers , 1990 .

[25]  R. Zwanzig.  From classical dynamics to continuous time random walks , 1983 .

[26]  Ioannis G. Kevrekidis,et al.  Nonlinear dimensionality reduction in molecular simulation: The diffusion map approach , 2011 .

[27]  Toni Giorgino,et al.  Identification of slow molecular order parameters for Markov model construction. , 2013, The Journal of chemical physics.

[28]  Mukund Balasubramanian,et al.  The Isomap Algorithm and Topological Stability , 2002, Science.

[29]  H. Berendsen,et al.  Essential dynamics of proteins , 1993, Proteins.

[30]  John D. Chodera,et al.  Long-Time Protein Folding Dynamics from Short-Time Molecular Dynamics Simulations , 2006, Multiscale Model. Simul..

[31]  I. Jolliffe.  Principal Component Analysis , 2002 .

[32]  A. Laio,et al.  Escaping free-energy minima , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[33]  Ricardo Vilalta,et al.  A Perspective View and Survey of Meta-Learning , 2002, Artificial Intelligence Review.

[34]  Vijay S Pande,et al.  Improvements in Markov State Model Construction Reveal Many Non-Native Interactions in the Folding of NTL9. , 2013, Journal of chemical theory and computation.

[35]  H. Kramers.  Brownian motion in a field of force and the diffusion model of chemical reactions , 1940 .

[36]  Frank Noé,et al.  Markov state models based on milestoning. , 2011, The Journal of chemical physics.

[37]  Dmitry P. Vetrov,et al.  Fast Adaptation in Generative Models with Generative Matching Networks , 2016, ICLR.

[38]  Berend Smit,et al.  Understanding molecular simulation: from algorithms to applications , 1996 .

[39]  Tom Schaul,et al.  Natural Evolution Strategies , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[40]  Eric Vanden-Eijnden,et al.  Transition Path Theory for Markov Jump Processes , 2009, Multiscale Model. Simul..