Error bounds for augmented truncation approximations of Markov chains via the perturbation method

Abstract Let P be the transition matrix of a positive recurrent Markov chain on the integers with invariant probability vector πT, and let (n)P̃ be a stochastic matrix, formed by augmenting the entries of the (n + 1) x (n + 1) northwest corner truncation of P arbitrarily, with invariant probability vector (n)πT. We derive computable V-norm bounds on the error between πT and (n)πT in terms of the perturbation method from three different aspects: the Poisson equation, the residual matrix, and the norm ergodicity coefficient, which we prove to be effective by showing that they converge to 0 as n tends to ∞ under suitable conditions. We illustrate our results through several examples. Comparing our error bounds with the ones of Tweedie (1998), we see that our bounds are more applicable and accurate. Moreover, we also consider possible extensions of our results to continuous-time Markov chains.

[1]  Mu-Fa Chen.  SINGLE BIRTH PROCESSES , 1999 .

[2]  Upendra Dave,et al.  Applied Probability and Queues , 1987 .

[3]  B. Heidergott,et al.  A Critical Account of Perturbation Analysis of Markov Chains , 2016, 1609.04138.

[4]  E. Altman,et al.  Perturbation analysis for denumerable Markov chains with application to queueing models , 2004, Advances in Applied Probability.

[5]  C. D. Meyer.  Sensitivity of the Stationary Distribution of a Markov Chain , 1994, SIAM J. Matrix Anal. Appl..

[6]  Djamil Aïssani,et al.  New perturbation bounds for denumerable Markov chains , 2010 .

[7]  E. Seneta.  Coefficients of ergodicity: structure and applications , 1979, Advances in Applied Probability.

[8]  Eugene Seneta,et al.  Augmented truncations of infinite stochastic matrices , 1987 .

[9]  A. Y. Mitrophanov,et al.  Sensitivity and convergence of uniformly ergodic Markov chains , 2005 .

[10]  Xi-Ren Cao,et al.  Perturbation analysis of discrete event dynamic systems , 1991 .

[11]  N. V. Kartashov.  Strong Stable Markov Chains , 1996 .

[12]  Yiqiang Q. Zhao,et al.  Censoring technique and numerical computations of invariant distribution for continuous-time Markov chains , 2015 .

[13]  Hiroyuki Masuyama,et al.  Error bounds for last-column-block-augmented truncations of block-structured Markov chains , 2016, 1601.03489.

[14]  Yuanyuan Liu,et al.  A unified perturbation analysis framework for countable Markov chains , 2017 .

[15]  Yiqiang Q. Zhao,et al.  Stochastic block–monotonicity in the approximation of the stationary distribution of infinite markov chains , 2000 .

[16]  Hiroyuki Masuyama,et al.  Error Bounds for Augmented Truncations of Discrete-Time Block-Monotone Markov Chains under Geometric Drift Conditions , 2015, Advances in Applied Probability.

[17]  Yuanyuan Liu.  Perturbation analysis for continuous-time Markov chains , 2015 .

[18]  Eugene Seneta,et al.  Computing the stationary distribution for infinite Markov chains , 1980 .

[19]  Xi-Ren Cao,et al.  Stochastic learning and optimization - A sensitivity-based approach , 2007, Annu. Rev. Control..

[20]  Marcel F. Neuts,et al.  Structured Stochastic Matrices of M/G/1 Type and Their Applications , 1989 .

[21]  N. Kartashov,et al.  Strongly stable Markov chains , 1986 .

[22]  C. D. Meyer,et al.  Comparison of perturbation bounds for the stationary distribution of a Markov chain , 2001 .

[23]  Nico M. van Dijk,et al.  Error Bounds and Comparison Results: The Markov Reward Approach For Queueing Networks , 2011 .

[24]  Hiroyuki Masuyama.  Error Bounds for Augmented Truncations of Discrete-Time Block-Monotone Markov Chains under Subgeometric Drift Conditions , 2016, SIAM J. Matrix Anal. Appl..

[25]  Yuanyuan Liu,et al.  Central limit theorems for ergodic continuous-time Markov chains with applications to single birth processes , 2015 .

[26]  Yuanyuan Liu,et al.  Perturbation Bounds for the Stationary Distributions of Markov Chains , 2012, SIAM J. Matrix Anal. Appl..

[27]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[28]  Sean P. Meyn,et al.  A Liapounov bound for solutions of the Poisson equation , 1996 .

[29]  Arie Hordijk,et al.  Series Expansions for Continuous-Time Markov Processes , 2010, Oper. Res..

[30]  Yuanyuan Liu,et al.  Augmented truncation approximations of discrete-time Markov chains , 2010, Oper. Res. Lett..

[31]  Richard L. Tweedie,et al.  Truncation approximations of invariant measures for Markov chains , 1998, Journal of Applied Probability.

[32]  Hiroyuki Masuyama,et al.  Continuous-time block-monotone Markov chains and their block-augmented truncations , 2015, 1511.04669.

[33]  Jeffrey J. Hunter,et al.  Stationary distributions and mean first passage times of perturbed Markov chains , 2005 .

[34]  Stephen J. Kirkland,et al.  On optimal condition numbers for Markov chains , 2008, Numerische Mathematik.