Entrywise perturbation theory and error analysis for Markov chains

SummaryGrassmann, Taksar, and Heyman introduced a variant of Gaussian climination for computing the steady-state vector of a Markov chain. In this paper we prove that their algorithm is stable, and that the problem itself is well-conditioned, in the sense of entrywise relative error. Thus the algorithm computes each entry of the steady-state vector with low relative error. Even the small steady-state probabilities are computed accurately. The key to our analysis is to focus on entrywise relative error in both the data and the computed solution, rather than making the standard assessments of error based on norms. Our conclusions do not depend on any Condition numbers for the problem.

[1]  James Hardy Wilkinson,et al.  Rounding errors in algebraic processes , 1964, IFIP Congress.

[2]  G. Stewart.  Introduction to matrix computations , 1973 .

[3]  R. Plemmons,et al.  LU decompositions of generalized diagonally dominant matrices , 1982 .

[4]  G. Stewart.  On the Structure of Nearly Uncoupled Markov Chains , 1983, Computer Performance and Reliability.

[5]  G. W. Stewart,et al.  Computable Error Bounds for Aggregated Markov Chains , 1983, JACM.

[6]  Gene H. Golub,et al.  Matrix computations , 1983 .

[7]  R. Plemmons,et al.  Comparison of Some Direct Methods for Computing Stationary Distributions of Markov Chains , 1984 .

[8]  Winfried K. Grassmann,et al.  Regenerative Analysis and Steady State Distributions for Markov Chains , 1985, Oper. Res..

[9]  Daniel P. Heyman,et al.  Further comparisons of direct methods for computing stationary distributions of Markov chains , 1987 .

[10]  C. D. Meyer,et al.  Derivatives and perturbations of eigenvectors , 1988 .

[11]  C. Blondia.  Finite-capacity Vacation Models With Nonrenewal Input , 1991 .

[12]  Jean C. Walrand.  Communication networks - a first course , 1991 .

[13]  G. Stewart,et al.  On a direct method for the solution of nearly uncoupled Markov chains , 1991 .

[14]  On the perturbation of Markov chains with nearly transient states , 1992 .

[15]  G. Stewart.  Gaussian Elimination, Perturbation Theory, and Markov Chains , 1993 .

[16]  Jesse L. Barlow.  Error bounds for the computation of null vectors with applications to Markov chains , 1993 .