Stationary Averaging for Multiscale Continuous Time Markov Chains Using Parallel Replica Dynamics

We propose two algorithms for simulating continuous time Markov chains in the presence of metastability. We show that the algorithms correctly estimate, under the ergodicity assumption, stationary averages of the process. Both algorithms, based on the idea of the parallel replica method, use parallel computing in order to explore metastable sets more efficiently. The algorithms require no assumptions on the Markov chains beyond ergodicity and the presence of identifiable metastability. In particular, there is no assumption on reversibility. For simpler illustration of the algorithms, we assume that a synchronous architecture is used throughout of the paper. We present error analyses, as well as numerical simulations on multi-scale stochastic reaction network models in order to demonstrate consistency of the method and its efficiency.

[1]  D. Gillespie.  Exact Stochastic Simulation of Coupled Chemical Reactions , 1977 .

[2]  J. Hammersley.  SIMULATION AND THE MONTE CARLO METHOD , 1982 .

[3]  C. Geyer.  Markov Chain Monte Carlo Maximum Likelihood , 1991 .

[4]  A. Voter.  Parallel replica method for dynamics of infrequent events , 1998 .

[5]  Muruhan Rathinam,et al.  Stiffness in stochastic chemically reacting systems: The implicit tau-leaping method , 2003 .

[6]  A. Doucet,et al.  Particle Motions in Absorbing Medium with Hard and Soft Obstacles , 2004 .

[7]  Michael W Deem,et al.  Parallel tempering: theory, applications, and new perspectives. , 2005, Physical chemistry chemical physics : PCCP.

[8]  Eric Vanden-Eijnden,et al.  Nested stochastic simulation algorithm for chemical kinetic systems with disparate rates. , 2005, The Journal of chemical physics.

[9]  Pablo A. Ferrari,et al.  Quasi Stationary Distributions and Fleming-Viot Processes in Countable Spaces , 2007 .

[10]  E. Seneta.  Non-negative Matrices and Markov Chains , 2008 .

[11]  Ioannis G Kevrekidis,et al.  A constrained approach to multiscale stochastic simulation of chemically reacting systems. , 2011, The Journal of chemical physics.

[12]  P. Dupuis,et al.  An infinite swapping approach to the rare-event sampling problem. , 2011, The Journal of chemical physics.

[13]  Danny Perez,et al.  A mathematical formalization of the parallel replica dynamics , 2011, Monte Carlo Methods Appl..

[14]  Paul Dupuis,et al.  On the Infinite Swapping Limit for Parallel Tempering , 2011, Multiscale Model. Simul..

[15]  Christof Schütte,et al.  Metastability and Markov State Models in Molecular Dynamics Modeling, Analysis , 2016 .

[16]  Gideon Simpson,et al.  The Parallel Replica Method for Simulating Long Trajectories of Markov Chains , 2014, 1401.4500.

[17]  Jose H. Blanchet,et al.  Theoretical analysis of a Stochastic Approximation approach for computing Quasi-Stationary distributions of general state space Markov chains , 2014 .

[18]  David Aristoff.  The parallel replica method for computing equilibrium averages of Markov chains , 2015, Monte Carlo Methods Appl..

[19]  Gideon Simpson,et al.  A generalized parallel replica dynamics , 2014, J. Comput. Phys..

[20]  Blas P. Uberuaga,et al.  The parallel replica dynamics method – Coming of age , 2015 .

[21]  Araz Hashemi,et al.  Stochastic averaging and sensitivity analysis for two scale reaction networks. , 2016, The Journal of chemical physics.

[22]  Ting Wang,et al.  Parallel replica dynamics method for bistable stochastic reaction networks: Simulation and sensitivity analysis. , 2017, The Journal of chemical physics.