Large‐scale asynchronous and distributed multidimensional replica exchange molecular simulations and efficiency analysis

We describe methods to perform replica exchange molecular dynamics (REMD) simulations asynchronously (ASyncRE). The methods are designed to facilitate large scale REMD simulations on grid computing networks consisting of heterogeneous and distributed computing environments as well as on homogeneous high‐performance clusters. We have implemented these methods on NSF (National Science Foundation) XSEDE (Extreme Science and Engineering Discovery Environment) clusters and BOINC (Berkeley Open Infrastructure for Network Computing) distributed computing networks at Temple University and Brooklyn College at CUNY (the City University of New York). They are also being implemented on the IBM World Community Grid. To illustrate the methods, we have performed extensive (more than 60 ms in aggregate) simulations for the beta‐cyclodextrin‐heptanoate host‐guest system in the context of one‐ and two‐dimensional ASyncRE, and we used the results to estimate absolute binding free energies using the binding energy distribution analysis method. We propose ways to improve the efficiency of REMD simulations: these include increasing the number of exchanges attempted after a specified molecular dynamics (MD) period up to the fast exchange limit and/or adjusting the MD period to allow sufficient internal relaxation within each thermodynamic state. Although ASyncRE simulations generally require long MD periods (>picoseconds) per replica exchange cycle to minimize the overhead imposed by heterogeneous computing networks, we found that it is possible to reach an efficiency similar to conventional synchronous REMD, by optimizing the combination of the MD period and the number of exchanges attempted per cycle. © 2015 Wiley Periodicals, Inc.

[1]  C. Brooks.  Computer simulation of liquids , 1989 .

[2]  T. Woolf,et al.  DYNAMIC REACTION PATHS AND RATES THROUGH IMPORTANCE-SAMPLED STOCHASTIC DYNAMICS , 1999 .

[3]  Yuko Okamoto,et al.  Ab Initio prediction of protein–ligand binding structures by replica‐exchange umbrella sampling simulations , 2011, J. Comput. Chem..

[4]  Enhancing QM/MM molecular dynamics sampling in explicit environments via an orthogonal-space-random-walk-based strategy. , 2011, The journal of physical chemistry. B.

[5]  Jianhan Chen,et al.  Efficiency of Adaptive Temperature-Based Replica Exchange for Sampling Large-Scale Protein Conformational Transitions. , 2013, Journal of chemical theory and computation.

[6]  M. Troyer,et al.  Optimized parallel tempering simulations of proteins. , 2006, The Journal of chemical physics.

[7]  Gerhard Hummer,et al.  Error and efficiency of simulated tempering simulations. , 2010, The Journal of chemical physics.

[8]  David E. Shaw,et al.  Zonal methods for the parallel execution of range-limited N-body simulations , 2007, J. Comput. Phys..

[9]  Bin W. Zhang,et al.  Efficient and verified simulation of a path ensemble for conformational change in a united-residue model of calmodulin , 2007, Proceedings of the National Academy of Sciences.

[10]  C. Zhang,et al.  Comparison of sampling efficiency between simulated tempering and replica exchange. , 2008, The Journal of chemical physics.

[11]  E. Vanden-Eijnden,et al.  String method for the study of rare events , 2002, cond-mat/0205527.

[12]  Lawrence R. Pratt,et al.  A statistical method for identifying transition states in high dimensional problems , 1986 .

[13]  Benoît Roux,et al.  Calculation of Free Energy Landscape in Multi-Dimensions with Hamiltonian-Exchange Umbrella Sampling on Petascale Supercomputer. , 2012, Journal of chemical theory and computation.

[14]  Wei Yang,et al.  Simulated scaling method for localized enhanced sampling and simultaneous "alchemical" free energy simulations: a general method for molecular mechanical, quantum mechanical, and quantum mechanical/molecular mechanical simulations. , 2007, The Journal of chemical physics.

[15]  M. Gilson,et al.  The statistical-thermodynamic basis for computation of binding affinities: a critical review. , 1997, Biophysical journal.

[16]  Markus Meuwly,et al.  Overcoming the Rare Event Sampling Problem in Biological Systems with Infinite Swapping. , 2013, Journal of chemical theory and computation.

[17]  R. Levy,et al.  Large scale affinity calculations of cyclodextrin host-guest complexes: Understanding the role of reorganization in the molecular recognition process. , 2013, Journal of chemical theory and computation.

[18]  Diwakar Shukla,et al.  To milliseconds and beyond: challenges in the simulation of protein folding. , 2013, Current opinion in structural biology.

[19]  Alan M. Ferrenberg,et al.  Optimized Monte Carlo data analysis. , 1989, Physical Review Letters.

[20]  Ulrich H E Hansmann,et al.  Velocity scaling for optimizing replica exchange molecular dynamics. , 2011, The Journal of chemical physics.

[21]  A Mitsutake,et al.  Generalized-ensemble algorithms for molecular simulations of biopolymers. , 2000, Biopolymers.

[22]  Daniel J. Sindhikara,et al.  Exchange Often and Properly in Replica Exchange Molecular Dynamics. , 2010, Journal of chemical theory and computation.

[23]  J. Mongan,et al.  Accelerated molecular dynamics: a promising and efficient simulation method for biomolecules. , 2004, The Journal of chemical physics.

[24]  L. Chong,et al.  Reaching biological timescales with all-atom molecular dynamics simulations. , 2010, Current opinion in pharmacology.

[25]  V. Pande,et al.  Multiplexed-replica exchange molecular dynamics method for protein folding simulation. , 2003, Biophysical journal.

[26]  G. Parisi,et al.  Simulated tempering: a new Monte Carlo scheme , 1992, hep-lat/9205018.

[27]  Yuji Sugita,et al.  Replica-exchange multicanonical algorithm and multicanonical replica-exchange method for simulating systems with rough energy landscape , 2000, cond-mat/0009119.

[28]  Mark J Abraham,et al.  Ensuring Mixing Efficiency of Replica-Exchange Molecular Dynamics Simulations. , 2008, Journal of chemical theory and computation.

[29]  Chris Oostenbrink,et al.  Optimization of replica exchange molecular dynamics by fast mimicking. , 2007, The Journal of chemical physics.

[30]  Federico D. Sacerdoti,et al.  Scalable Algorithms for Molecular Dynamics Simulations on Commodity Clusters , 2006, ACM/IEEE SC 2006 Conference (SC'06).

[31]  R. Swendsen,et al.  THE weighted histogram analysis method for free‐energy calculations on biomolecules. I. The method , 1992 .

[32]  Bernard R Brooks,et al.  pH replica‐exchange method based on discrete protonation states , 2011, Proteins.

[33]  C. Dellago,et al.  Transition path sampling and the calculation of rate constants , 1998 .

[34]  U H Hansmann,et al.  New Monte Carlo algorithms for protein folding. , 1999, Current opinion in structural biology.

[35]  Hugh Nymeyer,et al.  How Efficient Is Replica Exchange Molecular Dynamics? An Analytic Approach. , 2008, Journal of chemical theory and computation.

[36]  Martin Zacharias,et al.  Enhanced sampling of peptide and protein conformations using replica exchange simulations with a peptide backbone biasing‐potential , 2006, Proteins.

[37]  Alexander T. Hawk,et al.  Computation of transit times using the milestoning method with applications to polymer translocation. , 2013, The Journal of chemical physics.

[38]  Emilio Gallicchio,et al.  The AGBNP2 Implicit Solvation Model. , 2009, Journal of chemical theory and computation.

[39]  Shankar Kumar,et al.  Multidimensional free‐energy calculations using the weighted histogram analysis method , 1995, J. Comput. Chem..

[40]  Christoph Dellago,et al.  Efficient transition path sampling: Application to Lennard-Jones cluster rearrangements , 1998 .

[41]  Asim Okur,et al.  MDMS: Molecular Dynamics Meta-Simulator for evaluating exchange type sampling methods. , 2012, Chemical physics letters.

[42]  Ronald M. Levy,et al.  Prediction of SAMPL3 host-guest affinities with the binding energy distribution analysis method (BEDAM) , 2012, Journal of Computer-Aided Molecular Design.

[43]  Paul Tavan,et al.  Efficiency reduction and pseudo-convergence in replica exchange sampling of peptide folding–unfolding equilibria , 2008 .

[44]  Lingle Wang,et al.  On achieving high accuracy and reliability in the calculation of relative protein–ligand binding affinities , 2012, Proceedings of the National Academy of Sciences.

[45]  A. Roitberg,et al.  Coupling of replica exchange simulations to a non-Boltzmann structure reservoir. , 2007, The journal of physical chemistry. B.

[46]  R. Elber,et al.  Computing time scales from reaction coordinates by milestoning. , 2004, The Journal of chemical physics.

[47]  A. Laio,et al.  Free-energy landscape for beta hairpin folding from combined parallel tempering and metadynamics. , 2006, Journal of the American Chemical Society.

[48]  Z. Tan,et al.  Theory of binless multi-state free energy estimation with applications to protein-ligand binding. , 2012, The Journal of chemical physics.

[49]  Kai Wang,et al.  Identifying ligand binding sites and poses using GPU-accelerated Hamiltonian replica exchange molecular dynamics , 2013, Journal of Computer-Aided Molecular Design.

[50]  Adrian E. Roitberg,et al.  Multidimensional Replica Exchange Molecular Dynamics Yields a Converged Ensemble of an RNA Tetranucleotide , 2013, Journal of chemical theory and computation.

[51]  Chris Neale,et al.  Simulated Tempering Distributed Replica Sampling, Virtual Replica Exchange, and Other Generalized-Ensemble Methods for Conformational Sampling. , 2009, Journal of chemical theory and computation.

[52]  Adrian E Roitberg,et al.  Constant pH replica exchange molecular dynamics in biomolecules using a discrete protonation model. , 2010, Journal of chemical theory and computation.

[53]  R. Levy,et al.  Simple continuous and discrete models for simulating replica exchange simulations of protein folding. , 2008, The journal of physical chemistry. B.

[54]  Gerhard Hummer,et al.  Error and efficiency of replica exchange molecular dynamics simulations. , 2009, The Journal of chemical physics.

[55]  B. Berne,et al.  Replica exchange with solute scaling: a more efficient version of replica exchange with solute tempering (REST2). , 2011, The journal of physical chemistry. B.

[56]  Yuko Okamoto,et al.  Two‐dimensional replica‐exchange method for predicting protein–ligand binding structures , 2013, J. Comput. Chem..

[57]  Chun Wu,et al.  Convergence of replica exchange molecular dynamics. , 2005, The Journal of chemical physics.

[58]  Duncan Poole,et al.  Routine Microsecond Molecular Dynamics Simulations with AMBER on GPUs. 1. Generalized Born , 2012, Journal of chemical theory and computation.

[59]  S. Takada,et al.  On the Hamiltonian replica exchange method for efficient sampling of biomolecular systems: Application to protein structure prediction , 2002 .

[60]  Michael R. Shirts,et al.  Statistically optimal analysis of samples from multiple equilibrium states. , 2008, The Journal of chemical physics.

[61]  Emilio Gallicchio,et al.  Advances in all atom sampling methods for modeling protein-ligand binding affinities. , 2011, Current opinion in structural biology.

[62]  R. Nussinov,et al.  The role of dynamic conformational ensembles in biomolecular recognition. , 2009, Nature chemical biology.

[63]  Manish Parashar,et al.  Asynchronous replica exchange for molecular simulations , 2008, J. Comput. Chem..

[64]  U. Hansmann.  Parallel tempering algorithm for conformational studies of biological molecules , 1997, physics/9710041.

[65]  J. Andrew McCammon,et al.  Replica-Exchange Accelerated Molecular Dynamics (REXAMD) Applied to Thermodynamic Integration , 2008, Journal of chemical theory and computation.

[66]  Christian Bartels,et al.  Multidimensional adaptive umbrella sampling: Applications to main chain and side chain peptide conformations , 1997 .

[67]  Viktor Hornak,et al.  Improving Convergence of Replica-Exchange Simulations through Coupling to a High-Temperature Structure Reservoir. , 2007, Journal of chemical theory and computation.

[68]  Daniel M Zuckerman,et al.  Equilibrium sampling in biomolecular simulations. , 2011, Annual review of biophysics.

[69]  Arthur J. Olson,et al.  Virtual screening of integrase inhibitors by large scale binding free energy calculations: the SAMPL4 challenge , 2014, Journal of Computer-Aided Molecular Design.

[70]  Peter G. Bolhuis,et al.  A novel path sampling method for the calculation of rate constants , 2003 .

[71]  Eric Vanden-Eijnden,et al.  Infinite swapping replica exchange molecular dynamics leads to a simple simulation patch using mixture potentials. , 2012, The Journal of chemical physics.

[72]  Albert C. Pan,et al.  Finding transition pathways using the string method with swarms of trajectories. , 2008, The journal of physical chemistry. B.

[73]  Zhiqiang Tan,et al.  On a Likelihood Approach for Monte Carlo Integration , 2004 .

[74]  Wei Dai,et al.  A framework for flexible and scalable replica-exchange on production distributed CI , 2013, XSEDE.

[75]  A. Laio,et al.  Escaping free-energy minima , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[76]  I. Tavernelli,et al.  A Novel Hamiltonian Replica Exchange MD Protocol to Enhance Protein Conformational Space Sampling. , 2006, Journal of chemical theory and computation.

[77]  Anthony K. Felts,et al.  Temperature weighted histogram analysis method, replica exchange, and transition paths. , 2005, The journal of physical chemistry. B.

[78]  Frank Noé,et al.  An Introduction to Markov State Models and Their Application to Long Timescale Molecular Simulation , 2014, Advances in Experimental Medicine and Biology.

[79]  P. Dupuis,et al.  An infinite swapping approach to the rare-event sampling problem. , 2011, The Journal of chemical physics.

[80]  Y. Sugita,et al.  Replica-exchange molecular dynamics method for protein folding , 1999 .

[81]  A. Mark,et al.  Convergence and sampling efficiency in replica exchange simulations of peptide folding in explicit solvent. , 2007, The Journal of chemical physics.

[82]  Emilio Gallicchio,et al.  Recent theoretical and computational advances for modeling protein-ligand binding affinities. , 2011, Advances in protein chemistry and structural biology.

[83]  Daniel J. Sindhikara,et al.  Exchange frequency in replica exchange molecular dynamics. , 2008, The Journal of chemical physics.

[84]  Gerhard Hummer,et al.  Convergence and error estimation in free energy calculations using the weighted histogram analysis method , 2012, J. Comput. Chem..

[85]  B. Berne,et al.  Replica exchange with solute tempering: a method for sampling biological systems in explicit water. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[86]  Nadeem A. Vellore,et al.  An improved replica-exchange sampling method: temperature intervals with global energy reassignment. , 2007, The Journal of chemical physics.

[87]  Yuko Okamoto,et al.  Replica-exchange extensions of simulated tempering method. , 2004, The Journal of chemical physics.

[88]  R. Levy,et al.  Simulating replica exchange simulations of protein folding with a kinetic network model , 2007, Proceedings of the National Academy of Sciences.

[89]  William Sinko,et al.  w-REXAMD: A Hamiltonian Replica Exchange Approach to Improve Free Energy Calculations for Systems with Kinetically Trapped Conformations , 2013, Journal of chemical theory and computation.

[90]  R. Friesner,et al.  Evaluation and Reparametrization of the OPLS-AA Force Field for Proteins via Comparison with Accurate Quantum Chemical Calculations on Peptides† , 2001 .

[91]  C. Zhang,et al.  Enhanced sampling in generalized ensemble with large gap of sampling parameter: case study in temperature space random walk. , 2009, The Journal of chemical physics.

[92]  Régis Pomès,et al.  Distributed Replica Sampling. , 2006, Journal of chemical theory and computation.

[93]  David L Mobley,et al.  Alchemical free energy methods for drug discovery: progress and challenges. , 2011, Current opinion in structural biology.

[94]  Y. Sugita,et al.  Multidimensional replica-exchange method for free-energy calculations , 2000, cond-mat/0009120.

[95]  Emilio Gallicchio,et al.  Conformational Transitions and Convergence of Absolute Binding Free Energy Calculations. , 2012, Journal of chemical theory and computation.

[96]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[97]  Joseph A. Bank,et al.  Supporting Online Material Materials and Methods Figs. S1 to S10 Table S1 References Movies S1 to S3 Atomic-level Characterization of the Structural Dynamics of Proteins , 2022 .

[98]  Edward Lyman,et al.  A Second Look at Canonical Sampling of Biomolecules using Replica Exchange Simulation. , 2006, Journal of chemical theory and computation.

[99]  Wang,et al.  Replica Monte Carlo simulation of spin glasses. , 1986, Physical review letters.

[100]  Vijay S. Pande,et al.  Screen Savers of the World Unite! , 2000, Science.

[101]  Benoît Roux,et al.  Free Energy Perturbation Hamiltonian Replica-Exchange Molecular Dynamics (FEP/H-REMD) for Absolute Ligand Binding Free Energy Calculations. , 2010, Journal of chemical theory and computation.

[102]  W. L. Jorgensen,et al.  Development and Testing of the OPLS All-Atom Force Field on Conformational Energetics and Properties of Organic Liquids , 1996 .

[103]  Weihong Zhang,et al.  Replica exchange with guided annealing for accelerated sampling of disordered protein conformations , 2014, J. Comput. Chem..

[104]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[105]  Vijay S Pande,et al.  Convergence of folding free energy landscapes via application of enhanced sampling methods in a distributed computing environment. , 2008, The Journal of chemical physics.

[106]  Richard A. Friesner,et al.  Integrated Modeling Program, Applied Chemical Theory (IMPACT) , 2005, J. Comput. Chem..

[107]  Benoît Roux,et al.  Computation of Absolute Hydration and Binding Free Energy with Free Energy Perturbation Distributed Replica-Exchange Molecular Dynamics (FEP/REMD). , 2009, Journal of chemical theory and computation.

[108]  Michael R. Shirts,et al.  Replica exchange and expanded ensemble simulations as Gibbs sampling: simple improvements for enhanced mixing. , 2011, The Journal of chemical physics.

[109]  Gregory A Voth,et al.  Smart resolution replica exchange: an efficient algorithm for exploring complex energy landscapes. , 2007, The Journal of chemical physics.

[110]  Ronald M. Levy,et al.  AGBNP: An analytic implicit solvent model suitable for molecular dynamics simulations and high‐resolution modeling , 2004, J. Comput. Chem..

[111]  C. Geyer,et al.  Annealing Markov chain Monte Carlo with applications to ancestral inference , 1995 .

[112]  Emilio Gallicchio,et al.  The Binding Energy Distribution Analysis Method (BEDAM) for the Estimation of Protein-Ligand Binding Affinities. , 2010, Journal of chemical theory and computation.

[113]  Edward Lyman,et al.  Resolution exchange simulation. , 2005, Physical review letters.