Technical Note - Mean Drifts and the Non-Ergodicity of Markov Chains

THE STABILITY of many important systems is determined by the ergodicity of a related Markov chain. Since the steady-state distribution of a Markov chain is, however, often difficult to calculate, it is important to have simple criteria for the ergodicity or non-ergodicity of a chain. Let (Xn) (n = 0, 1, ***) denote an irreducible aperiodic Markov chain with state space IO, 1, * *. Let P = (pij) (i, j = 0, 1, *.. ) be its transition matrix and let pn = (p('7)) denote the nth power of P. It is well known that lim,.p(') = irj exists and is independent of i. Furthermore, either rj = 0 (j > 0), or rj > 0 (j > 0) and is the steady state distribution of the chain. The steady state probability vector also satisfies 7rP = 7r, that is