Computing Moments of First Passage Times to a Subset of States in Markov Chains

This paper presents a relatively efficient and accurate method to compute the moments of first passage times to a subset of states in finite ergodic Markov chains. With the proposed method, the moment computation problem is reduced to the solution of a linear system of equations with the right-hand side governed by a novel recurrence for computing the higher-order moments. We propose using a form of the Grassmann--Taksar--Heyman (GTH) algorithm to solve these linear equations. Due to the form of the linear systems involved, the proposed method does not suffer from the drawbacks associated with GTH in a row-wise sparse implementation.

[1]  Refael Hassin,et al.  Mean Passage Times and Nearly Uncoupled Markov Chains , 1992, SIAM J. Discret. Math..

[2]  Tugrul Dayar,et al.  On the Effects of Using the Grassmann-Taksar-Heyman Method in Iterative Aggregation-Disaggregation , 1996, SIAM J. Sci. Comput..

[3]  Ivo Marek,et al.  Iterative Aggregation/Disaggregation Methods for Computing Some Characteristics of Markov Chains , 2001, LSSC.

[4]  Daniel P. Heyman,et al.  Accurate Computation of the Fundamental Matrix of a Markov Chain , 1995, SIAM J. Matrix Anal. Appl..

[5]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[6]  Dianne P. O'Leary,et al.  What is Fundamental for Markov Chains: First Passage Times, Fundamental Matrices, and Group Generalized Inverses , 1995 .

[7]  R. Plemmons,et al.  Comparison of Some Direct Methods for Computing Stationary Distributions of Markov Chains , 1984 .

[8]  Ronald L. Graham,et al.  Concrete mathematics - a foundation for computer science , 1991 .

[9]  D. O’Leary,et al.  Overcoming Instability In Computing The Fundamental Matrix For A Markov Chain , 1998 .

[10]  Carl D. Meyer,et al.  Stochastic Complementation, Uncoupling Markov Chains, and the Theory of Nearly Reducible Systems , 1989, SIAM Rev..

[11]  Winfried K. Grassmann,et al.  Regenerative Analysis and Steady State Distributions for Markov Chains , 1985, Oper. Res..

[12]  John G. Kemeny,et al.  Finite Markov chains , 1960 .

[13]  J. Meyer.  The Role of the Group Generalized Inverse in the Theory of Finite Markov Chains , 1975 .

[14]  Vaidyanathan Ramaswami,et al.  Introduction to Matrix Analytic Methods in Stochastic Modeling , 1999, ASA-SIAM Series on Statistics and Applied Mathematics.

[15]  Edmundo de Souza e Silva,et al.  Calculating Cumulative Operational Time Distributions of Repairable Computer Systems , 1986, IEEE Transactions on Computers.

[16]  C. Paige,et al.  Computation of the stationary distribution of a markov chain , 1975 .