A Critical Account of Perturbation Analysis of Markov Chains

Perturbation analysis of Markov chains provides bounds on the effect that a change in a Markov transition matrix has on the corresponding stationary distribution. This paper compares and analyzes bounds found in the literature for finite and denumerable Markov chains and introduces new bounds based on series expansions. We discuss a series of examples to illustrate the applicability and numerical efficiency of the various bounds. Specifically, we address the question on how the bounds developed for finite Markov chains behave as the size of the system grows.In addition, we provide for the first time an analysis of the relative error of these bounds. For the case of a scaled perturbation we show that perturbation bounds can be used to analyze stability of a stable Markov chain with respect to perturbation with an unstable chain.

[1]  John G. Kemeny,et al.  Finite Markov Chains. , 1960 .

[2]  K. Thiruvengadam.  Queuing with Breakdowns , 1963 .

[3]  P. Naor,et al.  Some Queuing Problems with the Service Station Subject to Breakdown , 1963 .

[4]  P. Schweitzer.  Perturbation theory and finite Markov chains , 1968 .

[5]  John G. Kemeny,et al.  Finite Markov chains , 1960 .

[6]  CARL D. MEYER,et al.  The Condition of a Finite Markov Chain and Perturbation Bounds for the Limiting Probabilities , 1980, SIAM J. Algebraic Discret. Methods.

[7]  Francois Baccelli,et al.  QUEUEING SYSTEMS WITH BREAKDOWNS AND DATA BASE MODELLING. , 1981 .

[8]  J. Hunter.  Generalized inverses and their application to applied probability problems , 1982 .

[9]  L. V. D. Heyden,et al.  Perturbation bounds for the stationary probabilities of a finite Markov chain , 1984 .

[10]  N. Kartashov,et al.  Strongly stable Markov chains , 1986 .

[11]  Vladimir V. Anisimov,et al.  Estimates for the deviations of the transition characteristics of nonhomogeneous markov processes , 1988 .

[12]  C. O'Cinneide.  Entrywise perturbation theory and error analysis for Markov chains , 1993 .

[13]  Ilse C. F. Ipsen,et al.  Uniform Stability of Markov Chains , 1994, SIAM J. Matrix Anal. Appl..

[14]  N. V. Kartashov.  Strong Stable Markov Chains , 1996 .

[15]  Gang George Yin,et al.  Asymptotic Expansions of Singularly Perturbed Systems Involving Rapidly Fluctuating Markov Chains , 1996, SIAM J. Appl. Math..

[16]  Xiuli Chao,et al.  Reliability analysis of M/G/1 queueing systems with server breakdowns and vacations , 1997, Journal of Applied Probability.

[17]  G. Yin,et al.  Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach , 1997 .

[18]  Masaaki Kijima,et al.  Markov processes for stochastic modeling , 1997 .

[19]  Xi-Ren Cao.  The Maclaurin series for performance functions of Markov chains , 1998, Advances in Applied Probability.

[20]  Weiguo Gao,et al.  Blockwise Perturbation Theory for Markov Chains , 1998, SIAM J. Matrix Anal. Appl..

[21]  Bryan L. Shader,et al.  Applications of Paz's inequality to perturbation bounds for Markov chains , 1998 .

[22]  Gang George Yin,et al.  Singularly Perturbed Discrete-Time Markov Chains , 2000, SIAM J. Appl. Math..

[23]  Ger Koole,et al.  ON DEVIATION MATRICES FOR BIRTH–DEATH PROCESSES , 2001, Probability in the Engineering and Informational Sciences.

[24]  Quanlin Li,et al.  Reliability Analysis of the Retrial Queue with Server Breakdowns and Repairs , 2001, Queueing Syst. Theory Appl..

[25]  Andrew L. Rukhin,et al.  Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach , 2001, Technometrics.

[26]  C. D. Meyer,et al.  Comparison of perturbation bounds for the stationary distribution of a Markov chain , 2001 .

[27]  Stephen J. Kirkland.  On a Question Concerning Condition Numbers for Markov Chains , 2002, SIAM J. Matrix Anal. Appl..

[28]  Pauline Coolen-Schrijner,et al.  THE DEVIATION MATRIX OF A CONTINUOUS-TIME MARKOV CHAIN , 2002, Probability in the Engineering and Informational Sciences.

[29]  A. Hordijk,et al.  Taylor series expansions for stationary Markov chains , 2003, Advances in Applied Probability.

[30]  M. Neumann,et al.  Improved bounds for a condition number for Markov chains , 2004 .

[31]  A. Y. Mitrophanov,et al.  Sensitivity and convergence of uniformly ergodic Markov chains , 2005 .

[32]  Mark Borodovsky,et al.  SENSITIVITY OF HIDDEN MARKOV MODELS , 2005 .

[33]  Djamil Aïssani,et al.  Performance analysis approximation in a queueing system of type M/G/1 , 2006, Math. Methods Oper. Res..

[34]  A. Hordijk,et al.  SERIES EXPANSIONS FOR FINITE-STATE MARKOV CHAINS , 2007, Probability in the Engineering and Informational Sciences.

[35]  Aniruddha Datta,et al.  Robust Intervention in Probabilistic Boolean Networks , 2007, ACC.

[36]  Stephen J. Kirkland,et al.  On optimal condition numbers for Markov chains , 2008, Numerische Mathematik.

[37]  D. Aïssani,et al.  Strong Stability of the Batch Arrival Queueing Systems , 2009 .

[38]  C. Andrieu,et al.  The pseudo-marginal approach for efficient Monte Carlo computations , 2009, 0903.5480.

[39]  Arie Hordijk,et al.  Series Expansions for Continuous-Time Markov Processes , 2010, Oper. Res..

[40]  Matthew O. Jackson,et al.  Naïve Learning in Social Networks and the Wisdom of Crowds , 2010 .

[41]  Karim Abbas,et al.  Structural Perturbation Analysis of a Single Server Queue with Breakdowns , 2010 .

[42]  Djamil Aïssani,et al.  New perturbation bounds for denumerable Markov chains , 2010 .

[43]  Djamil Aïssani,et al.  Error bounds on practical approximation for two tandem queue with blocking and non-preemptive priority , 2011, Comput. Math. Appl..

[44]  Ilse C. F. Ipsen,et al.  Ergodicity Coefficients Defined by Vector Norms , 2011, SIAM J. Matrix Anal. Appl..

[45]  Yuanyuan Liu,et al.  Perturbation Bounds for the Stationary Distributions of Markov Chains , 2012, SIAM J. Matrix Anal. Appl..

[46]  Luca Bortolussi,et al.  Bounds on the deviation of discrete-time Markov chains from their mean-field model , 2013, Perform. Evaluation.

[47]  Michael M. Wolf,et al.  Perturbation bounds for quantum Markov processes and their fixed points , 2012, 1210.1171.

[48]  H. Caswell.  Sensitivity analysis of discrete Markov chains via matrix calculus , 2013 .

[49]  Wen Li,et al.  On Perturbation Bounds for the Joint Stationary Distribution of Multivariate Markov Chain Models , 2013 .

[50]  Perturbation results for comparison of Markov models , 2013 .

[51]  Michael Ghil,et al.  Rough parameter dependence in climate models and the role of Ruelle-Pollicott resonances , 2014, Proceedings of the National Academy of Sciences.

[52]  Giacomo Como,et al.  Robustness of Large-Scale Stochastic Matrices to Localized Perturbations , 2013, IEEE Transactions on Network Science and Engineering.

[53]  Pierre Alquier,et al.  Noisy Monte Carlo: convergence of Markov chains with approximate transition kernels , 2014, Statistics and Computing.