Stochastic Optimal Control for Collective Variable Free Sampling of Molecular Transition Paths

We consider the problem of sampling transition paths between two given metastable states of a molecular system, e.g. a folded and unfolded protein or products and reactants of a chemical reaction. Due to the existence of high energy barriers separating the states, these transition paths are unlikely to be sampled with standard Molecular Dynamics (MD) simulation. Traditional methods to augment MD with a bias potential to increase the probability of the transition rely on a dimensionality reduction step based on Collective Variables (CVs). Unfortunately, selecting appropriate CVs requires chemical intuition and traditional methods are therefore not always applicable to larger systems. Additionally, when incorrect CVs are used, the bias potential might not be minimal and bias the system along dimensions irrelevant to the transition. Showing a formal relation between the problem of sampling molecular transition paths, the Schr\"odinger bridge problem and stochastic optimal control with neural network policies, we propose a machine learning method for sampling said transitions. Unlike previous non-machine learning approaches our method, named PIPS, does not depend on CVs. We show that our method successful generates low energy transitions for Alanine Dipeptide as well as the larger Polyproline and Chignolin proteins.

[1]  T. Jaakkola,et al.  Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations , 2022, Trans. Mach. Learn. Res..

[2]  Neil D. Lawrence,et al.  Solving Schrödinger Bridges via Maximum Likelihood , 2021, Entropy.

[3]  Valentin De Bortoli,et al.  Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling , 2021, NeurIPS.

[4]  C. Camilloni,et al.  How to Determine Accurate Conformational Ensembles by Metadynamics Metainference: A Chignolin Study Case , 2021, Frontiers in Molecular Biosciences.

[5]  J. P. Garrahan,et al.  Reinforcement learning of rare diffusive dynamics , 2021, The Journal of chemical physics.

[6]  Jian Tang,et al.  Learning Gradient Fields for Molecular Conformation Generation , 2021, ICML.

[7]  B. Ensing,et al.  Discovering Collective Variables of Molecular Transitions via Genetic Algorithms and Neural Networks , 2021, Journal of chemical theory and computation.

[8]  Victor Garcia Satorras,et al.  E(n) Equivariant Graph Neural Networks , 2021, ICML.

[9]  Klaus-Robert Müller,et al.  Machine Learning Force Fields , 2020, Chemical reviews.

[10]  Ron O. Dror,et al.  Molecular Dynamics Simulation for All , 2018, Neuron.

[11]  Mohammad M. Sultan,et al.  Transferable Neural Networks for Enhanced Sampling of Protein Dynamics. , 2018, Journal of chemical theory and computation.

[12]  Mark E Tuckerman,et al.  Stochastic Neural Network Approach for Learning High-Dimensional Free Energy Surfaces. , 2017, Physical review letters.

[13]  Vijay S. Pande,et al.  OpenMM 7: Rapid development of high performance algorithms for molecular dynamics , 2016, bioRxiv.

[14]  M. Welling,et al.  Group Equivariant Convolutional Networks , 2016, ICML.

[15]  Hilbert J. Kappen,et al.  Adaptive Importance Sampling for Control and Inference , 2015, ArXiv.

[16]  Giovanni Bussi,et al.  Free‐Energy Calculations with Metadynamics: Theory and Practice , 2015 .

[17]  Tryphon T. Georgiou,et al.  On the Relation Between Optimal Transport and Schrödinger Bridges: A Stochastic Control Viewpoint , 2014, J. Optim. Theory Appl..

[18]  Christophe Chipot,et al.  The Adaptive Biasing Force Method: Everything You Always Wanted To Know but Were Afraid To Ask , 2014, The journal of physical chemistry. B.

[19]  Søren Enemark,et al.  β-hairpin forms by rolling up from C-terminal: Topological guidance of early folding dynamics , 2012, Scientific Reports.

[20]  R. Dror,et al.  How Fast-Folding Proteins Fold , 2011, Science.

[21]  John D. Chodera,et al.  Using Nonequilibrium Fluctuation Theorems to Understand and Correct Errors in Equilibrium and Nonequ , 2011, 1107.2967.

[22]  Ryuhei Harada,et al.  Exploring the folding free energy landscape of a β-hairpin miniprotein, chignolin, using multiscale free energy landscape calculation method. , 2011, The journal of physical chemistry. B.

[23]  R. Dror,et al.  Improved side-chain torsion potentials for the Amber ff99SB protein force field , 2010, Proteins.

[24]  M. Parrinello,et al.  Well-tempered metadynamics: a smoothly converging and tunable free-energy method. , 2008, Physical review letters.

[25]  M. Parrinello,et al.  Accurate sampling using Langevin dynamics. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[26]  Makoto Taiji,et al.  Folding Dynamics of 10‐Residue β‐Hairpin Peptide Chignolin , 2007 .

[27]  H. Kappen An introduction to stochastic control theory, path integrals and reinforcement learning , 2007 .

[28]  Kentaro Shimizu,et al.  Folding free‐energy landscape of a 10‐residue mini‐protein, chignolin , 2006, FEBS letters.

[29]  M. Seibert,et al.  Reproducible polypeptide folding and structure prediction using molecular dynamics simulations. , 2005, Journal of molecular biology.

[30]  H. Kappen Path integrals and symmetry breaking for optimal control theory , 2005, physics/0505066.

[31]  Shinya Honda,et al.  10 residue folded peptide designed by segment statistics. , 2004, Structure.

[32]  A. Laio,et al.  Escaping free-energy minima , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[33]  Eric F Darve,et al.  Calculating free energies using average force , 2001 .

[34]  G. Henkelman,et al.  A climbing image nudged elastic band method for finding saddle points and minimum energy paths , 2000 .

[35]  D. Landau,et al.  Efficient, multiple-range random walk algorithm to calculate the density of states. , 2000, Physical review letters.

[36]  C. Dellago,et al.  Reaction coordinates of biomolecular isomerization. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[37]  Michiel Sprik,et al.  Free energy from constrained molecular dynamics , 1998 .

[38]  C. Dellago,et al.  Transition path sampling and the calculation of rate constants , 1998 .

[39]  A. Voter A method for accelerating the molecular dynamics simulation of infrequent events , 1997 .

[40]  Berend Smit,et al.  Understanding molecular simulation: from algorithms to applications , 1996 .

[41]  T. Darden,et al.  A smooth particle mesh Ewald method , 1995 .

[42]  Grubmüller,et al.  Predicting slow structural transitions in macromolecular systems: Conformational flooding. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[43]  Andrew E. Torda,et al.  Local elevation: A method for improving the searching properties of molecular dynamics simulation , 1994, J. Comput. Aided Mol. Des..

[44]  G. Ciccotti,et al.  Constrained reaction coordinate dynamics for the simulation of rare events , 1989 .

[45]  G. Torrie,et al.  Nonphysical sampling distributions in Monte Carlo free-energy estimation: Umbrella sampling , 1977 .

[46]  W. T. Martin,et al.  Transformations of Weiner Integrals Under Translations , 1944 .

[47]  David Chandler,et al.  Transition path sampling: throwing ropes over rough mountain passes, in the dark. , 2002, Annual review of physical chemistry.

[48]  E. Schrödinger Sur la théorie relativiste de l'électron et l'interprétation de la mécanique quantique , 1932 .