Parallel computing for Markov chains with islands and ports

We develop an algorithm to calculate invariant distributions of large Markov chains whose state spaces are partitioned into “islands” and “ports”. An island is a group of states (cluster) with potentially many connections inside of the island but a relatively small number of connections between islands. The states connecting different islands are called ports. Our algorithm is developed in the framework of the “state reduction approach”, but the special structure of the state space allows calculation of the invariant distribution to be done in parallel. Additional problems such as computation of fundamental matrices and optimal stopping problems are also analyzed for such Markov chains.

[1]  Konstantin Avrachenkov,et al.  A survey on distributed approaches to graph based reputation measures , 2007, Valuetools 2007.

[2]  Amy Nicole Langville,et al.  Google's PageRank and beyond - the science of search engine rankings , 2006 .

[3]  Nicolas Parisey,et al.  Interpretation and approximation tools for big, dense Markov chain transition matrices in population genetics , 2015, Algorithms for Molecular Biology.

[4]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[5]  C. D. Meyer,et al.  Markov chain sensitivity measured by mean first passage times , 2000 .

[6]  Isaac Sonin,et al.  Continue, quit, restart probability model , 2016, Ann. Oper. Res..

[7]  Konstantin Avrachenkov,et al.  Analytic Perturbation Theory and Its Applications , 2013 .

[8]  C. D. Meyer,et al.  Comparison of perturbation bounds for the stationary distribution of a Markov chain , 2001 .

[9]  Michael N. Katehakis,et al.  A SUCCESSIVE LUMPING PROCEDURE FOR A CLASS OF MARKOV CHAINS , 2012, Probability in the Engineering and Informational Sciences.

[10]  E. Presman.  Solution of Optimal Stopping Problem Based on a Modification of Payoff Function , 2014 .

[11]  Refael Hassin,et al.  Mean Passage Times and Nearly Uncoupled Markov Chains , 1992, SIAM J. Discret. Math..

[12]  Carl Dean Meyer,et al.  Stochastic Data Clustering , 2010, SIAM J. Matrix Anal. Appl..

[13]  C. O'Cinneide.  Entrywise perturbation theory and error analysis for Markov chains , 1993 .

[14]  Michael N. Katehakis,et al.  A comparative analysis of the successive lumping and the lattice path counting algorithms , 2015, Journal of Applied Probability.

[15]  William J. Stewart,et al.  Probability, Markov Chains, Queues, and Simulation: The Mathematical Basis of Performance Modeling , 2009 .

[16]  Isaac Sonin,et al.  The Elimination algorithm for the problem of optimal stopping , 1999, Math. Methods Oper. Res..

[17]  Ning Liu,et al.  Markov Chains and Spectral Clustering , 2010, PERFORM.

[18]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[19]  Carl D. Meyer,et al.  Stochastic Complementation, Uncoupling Markov Chains, and the Theory of Nearly Reducible Systems , 1989, SIAM Rev..

[20]  Daniel P. Heyman,et al.  Accurate Computation of the Fundamental Matrix of a Markov Chain , 1995, SIAM J. Matrix Anal. Appl..

[21]  Isaac M. Sonin,et al.  The State Reduction and Related Algorithms and Their Applications to the Study of Markov Chains, Graph Theory, and the Optimal Stopping Problem , 1999 .

[22]  Tugrul Dayar,et al.  On the Effects of Using the Grassmann-Taksar-Heyman Method in Iterative Aggregation-Disaggregation , 1996, SIAM J. Sci. Comput..

[23]  Anna Gambin,et al.  A New Combinatorial Algorithm for Large Markov Chains (Extended Abstract) , 2001, CASC.

[24]  Karl Gustafson,et al.  Why the Kemeny Time is a constant , 2015, 1510.00456.

[25]  Winfried K. Grassmann,et al.  Matrix Analytic Methods , 2000 .

[26]  Isaac M. Sonin,et al.  The Optimal Stopping of a Markov Chain and Recursive Solution of Poisson and Bellman Equations , 2006 .

[27]  Winfried K. Grassmann,et al.  Regenerative Analysis and Steady State Distributions for Markov Chains , 1985, Oper. Res..

[28]  Gene H. Golub,et al.  Exploiting the Block Structure of the Web for Computing , 2003 .

[29]  Theodore J. Sheskin,et al.  Technical Note - A Markov Chain Partitioning Algorithm for Computing Steady State Probabilities , 1985, Oper. Res..

[30]  L. V. D. Heyden,et al.  Perturbation bounds for the stationary probabilities of a finite Markov chain , 1984 .

[31]  Isaac Sonin,et al.  Recursive Algorithm for the Fundamental/Group Inverse Matrix of a Markov Chain from an Explicit Formula , 2001, SIAM J. Matrix Anal. Appl..

[32]  W. Stewart,et al.  ITERATIVE METHODS FOR COMPUTING STATIONARY DISTRIBUTIONS OF NEARLY COMPLETELY DECOMPOSABLE MARKOV CHAINS , 1984 .

[33]  I. Sonin.  A generalized Gittins index for a Markov chain and its recursive calculation , 2008 .

[34]  Konstantin Avrachenkov,et al.  A Singular Perturbation Approach for Choosing the PageRank Damping Factor , 2008, Internet Math..

[35]  Wilfried N. Gansterer,et al.  Performance Evaluation of Computer and Communication Systems. Milestones and Future Challenges , 2011 .