GTH Algorithm, Censored Markov Chains, and $RG$-Factorization

In this paper, we provide a review on the GTH algorithm, which is a numerically stable algorithm for computing stationary probabilities of a Markov chain. Mathematically the GTH algorithm is an rearrangement of Gaussian elimination, and therefore they are mathematically equivalent. All components in the GTH algorithm can be interpreted probabilistically based on the censoring concept and each elimination in the GTH algorithm leads to a censored Markov chain. The RG-factorization is a counterpart to the LU-decomposition for Gaussian elimination. The censored Markov chain can also be treated as an extended version of the GTH algorithm for a system consisting of infinitely many linear equations. The censored Markov chain produces a minimal error for approximating the original chain under the l1-norm.

[1]  Hiroyuki Masuyama,et al.  Error bounds for last-column-block-augmented truncations of block-structured Markov chains , 2016, 1601.03489.

[2]  Winfried K. Grassmann,et al.  Regenerative Analysis and Steady State Distributions for Markov Chains , 1985, Oper. Res..

[3]  On a Decomposition for In nite Transition , 1997 .

[4]  Paul Lévy.  Processus markoviens et stationnaires. Cas dénombrable , 1958 .

[5]  Quan-Lin Li,et al.  β-Invariant Measures for Transition Matrices of GI/M/1 Type , 2003 .

[6]  J. Kemeny,et al.  Denumerable Markov chains , 1969 .

[7]  Yiqiang Q. Zhao,et al.  Heterogeneous Multiserver Queues With General Input , 1997 .

[8]  Quan-Lin Li,et al.  THE RG-FACTORIZATION IN BLOCK-STRUCTURED MARKOV RENEWAL PROCESSES , 2004 .

[9]  Eugene Seneta,et al.  Computing the stationary distribution for infinite Markov chains , 1980 .

[10]  Tugrul Dayar,et al.  On the Effects of Using the Grassmann-Taksar-Heyman Method in Iterative Aggregation-Disaggregation , 1996, SIAM J. Sci. Comput..

[11]  Wei Li,et al.  Censoring, Factorizations, and Spectral Analysis for Transition Matrices with Block-Repeating Entries , 2001 .

[12]  Eugene Seneta,et al.  Augmented truncations of infinite stochastic matrices , 1987 .

[13]  P. Levy,et al.  Complément à l'étude des processus de Markoff , 1952 .

[14]  Yuanyuan Liu,et al.  Error bounds for augmented truncation approximations of Markov chains via the perturbation method , 2018, Advances in Applied Probability.

[15]  C. O'Cinneide.  Entrywise perturbation theory and error analysis for Markov chains , 1993 .

[16]  Isaac Sonin,et al.  Recursive Algorithm for the Fundamental/Group Inverse Matrix of a Markov Chain from an Explicit Formula , 2001, SIAM J. Matrix Anal. Appl..

[17]  Jürg Kohlas.  Numerical computation of mean passage times and absorption probabilities in Markov and Semi-Markov models , 1986, Z. Oper. Research.

[18]  G. Stewart.  Gaussian Elimination, Perturbation Theory, and Markov Chains , 1993 .

[19]  Quan-Lin Li,et al.  A CONSTRUCTIVE METHOD FOR FINDING fl-INVARIANT MEASURES FOR TRANSITION MATRICES OF M=G=1 TYPE , 2002 .

[20]  Danielle Liu,et al.  The censored Markov chain and the best augmentation , 1996, Journal of Applied Probability.

[21]  Paul Lévy.  Systèmes markoviens et stationnaires. Cas dénombrable , 1951 .

[22]  Bradford R. Cain,et al.  Approximating Countable Markov Chains , 1974 .

[23]  Jeffrey J Hunter.  The computation of the mean first passage times for Markov chains , 2018, Linear Algebra and its Applications.

[24]  Winfried K. Grassmann,et al.  Equilibrium distribution of block-structured Markov chains with repeating rows , 1990, Journal of Applied Probability.

[25]  Nail Akar,et al.  Computing Moments of First Passage Times to a Subset of States in Markov Chains , 2005, SIAM J. Matrix Anal. Appl..

[26]  Daniel P. Heyman,et al.  Further comparisons of direct methods for computing stationary distributions of Markov chains , 1987 .

[27]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[28]  Richard L. Tweedie,et al.  Truncation approximations of invariant measures for Markov chains , 1998, Journal of Applied Probability.

[29]  Winfried K. Grassmann,et al.  Rounding errors in certain algorithms involving Markov chains , 1993, TOMS.