Perturbation analysis for continuous-time Markov chains

We investigate perturbation for continuous-time Markov chains (CTMCs) on a countable state space. Explicit bounds on ΔD and D are derived in terms of a drift condition, where Δ and D represent the perturbation of the intensity matrices and the deviation matrix, respectively. Moreover, we obtain perturbation bounds on the stationary distributions, which extends the results by Liu (2012) for uniformly bounded CTMCs to general (possibly unbounded) CTMCs. Our arguments are mainly based on the technique of augmented truncations.

[1]  R. Tweedie.  Perturbations of countable Markov chains and processes , 1980, Advances in Applied Probability.

[2]  Mu-Fa Chen,et al.  From Markov Chains to Non-Equilibrium Particle Systems , 1992 .

[3]  S. Meyn,et al.  Stability of Markovian processes III: Foster–Lyapunov criteria for continuous-time processes , 1993, Advances in Applied Probability.

[4]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[5]  S. Meyn,et al.  Computable exponential convergence rates for stochastically ordered Markov processes , 1996 .

[6]  N. V. Kartashov.  Strong Stable Markov Chains , 1996 .

[7]  Sean P. Meyn,et al.  A Liapounov bound for solutions of the Poisson equation , 1996 .

[8]  Danielle Liu,et al.  The censored Markov chain and the best augmentation , 1996, Journal of Applied Probability.

[9]  J. Rosenthal,et al.  Geometric Ergodicity and Hybrid Markov Chains , 1997 .

[10]  Gareth O. Roberts,et al.  Convergence Properties of Perturbed Markov Chains , 1998, Journal of Applied Probability.

[11]  Richard L. Tweedie,et al.  Truncation approximations of invariant measures for Markov chains , 1998, Journal of Applied Probability.

[12]  Anyue Chen,et al.  Strong ergodicity of monotone transition functions , 2001 .

[13]  Gareth O. Roberts,et al.  A note on geometric ergodicity and floating-point roundoff error , 2001 .

[14]  Pauline Coolen-Schrijner,et al.  THE DEVIATION MATRIX OF A CONTINUOUS-TIME MARKOV CHAIN , 2002, Probability in the Engineering and Informational Sciences.

[15]  Yu-hui Zhang,et al.  Exponential ergodicity for single-birth processes , 2004 .

[16]  Yong-Hua Mao,et al.  Ergodic degrees for continuous-time Markov chains , 2004 .

[17]  E. Altman,et al.  Perturbation analysis for denumerable Markov chains with application to queueing models , 2004, Advances in Applied Probability.

[18]  Jeffrey J. Hunter,et al.  Stationary distributions and mean first passage times of perturbed Markov chains , 2005 .

[19]  A. Yu. Mitrophanov,et al.  Stability Estimates for Finite Homogeneous Continuous-time Markov Chains , 2006 .

[20]  M. Neumann,et al.  Transition matrices for well-conditioned Markov chains , 2007 .

[21]  Yiqiang Q. Zhao,et al.  COMPUTABLE STRONGLY ERGODIC RATES OF CONVERGENCE FOR CONTINUOUS-TIME MARKOV CHAINS , 2008, The ANZIAM Journal.

[22]  Arie Hordijk,et al.  Series Expansions for Continuous-Time Markov Processes , 2010, Oper. Res..

[23]  Yuanyuan Liu,et al.  Augmented truncation approximations of discrete-time Markov chains , 2010, Oper. Res. Lett..

[24]  Yiqiang Q. Zhao,et al.  Subgeometric ergodicity for continuous-time Markov chains , 2010 .

[25]  Djamil Aïssani,et al.  New perturbation bounds for denumerable Markov chains , 2010 .

[26]  Yuanyuan Liu,et al.  Perturbation Bounds for the Stationary Distributions of Markov Chains , 2012, SIAM J. Matrix Anal. Appl..

[27]  Alexander I. Zeifman,et al.  Perturbation Bounds for M t /M t /N Queue with Catastrophes , 2012 .

[28]  Richard L. Tweedie,et al.  Convergence of Invariant Measures of Truncation Approximations to Markov Processes , 2012 .

[29]  Yuanyuan Liu,et al.  Poisson's equation for discrete-time quasi-birth-and-death processes , 2013, Perform. Evaluation.

[30]  Yuanyuan Liu,et al.  Deviation matrix and asymptotic variance for GI/M/1-type Markov chains , 2014 .

[31]  Yuanyuan Liu,et al.  Poisson’s equation for discrete-time single-birth processes , 2014 .