Stability of Markovian processes I: criteria for discrete-time Chains

In this paper we connect various topological and probabilistic forms of stability for discrete-time Markov chains. These include tightness on the one hand and Harris recurrence and ergodicity on the other. We show that these concepts of stability are largely equivalent for a major class of chains (chains with continuous components), or if the state space has a sufficiently rich class of appropriate sets (‘petite sets'). We use a discrete formulation of Dynkin's formula to establish unified criteria for these stability concepts, through bounding of moments of first entrance times to petite sets. This gives a generalization of Lyapunov–Foster criteria for the various stability conditions to hold. Under these criteria, ergodic theorems are shown to be valid even in the non-irreducible case. These results allow a more general test function approach for determining rates of convergence of the underlying distributions of a Markov chain, and provide strong mixing results and new versions of the central limit theorem and the law of the iterated logarithm.

[1]  F. G. Foster On the Stochastic Matrices Associated with Certain Queuing Processes , 1953 .

[2]  Kai Lai Chung,et al.  Markov Chains with Stationary Transition Probabilities , 1961 .

[3]  John Lamperti,et al.  Criteria for the recurrence or transience of stochastic process. I , 1960 .

[4]  David Vere-Jones SOME LIMIT THEOREMS FOR EVANESCENT PROCESSES , 1969 .

[5]  S. Foguel POSITIVE OPERATORS ON C(X) , 1969 .

[6]  G. P. Szegö,et al.  Stability theory of dynamical systems , 1970 .

[7]  S. Orey Lecture Notes on Limit Theorems for Markov Chain Transition Probabilities , 1971 .

[8]  D. Vere-Jones Markov Chains , 1972, Nature.

[9]  R. Cogburn A Uniform Theory for Sums of Markov Chain Transition Probabilities , 1975 .

[10]  R. Tweedie Sufficient conditions for ergodicity and recurrence of Markov chains on a general state space , 1975 .

[11]  R. Tweedie,et al.  R -Theory for Markov Chains on a Topological State Space I , 1975 .

[12]  R. Tweedie,et al.  R-theory for Markov chains on a topological state Space. II , 1976 .

[13]  R. Tweedie Criteria for classifying general Markov chains , 1976, Advances in Applied Probability.

[14]  E. Nummelin,et al.  A splitting technique for Harris recurrent Markov chains , 1978 .

[15]  R. Tweedie,et al.  Geometric Ergodicity and R-positivity for General Markov Chains , 1978 .

[16]  K. Athreya,et al.  A New Approach to the Limit Theory of Recurrent Markov Chains , 1978 .

[17]  R. Tweedie,et al.  Markov Chains with Continuous Components , 1979 .

[18]  R. Khasminskii Stochastic Stability of Differential Equations , 1980 .

[19]  Sheldon M. Ross,et al.  Stochastic Processes , 2018, Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics.

[20]  E. Nummelin,et al.  Geometric ergodicity of Harris recurrent Marcov chains with applications to renewal theory , 1982 .

[21]  R. L. Tweedie,et al.  Probability, Statistics and Analysis: Criteria for rates of convergence of Markov chains, with application to queueing and storage theory , 1983 .

[22]  E. Nummelin General irreducible Markov chains and non-negative operators: List of symbols and notation , 1984 .

[23]  K. Athreya,et al.  Mixing properties of harris chains and autoregressive processes , 1986, Journal of Applied Probability.

[24]  R. Tweedie Invariant measures for Markov chains with no irreducibility assumptions , 1988 .

[25]  Sean P. Meyn,et al.  Stability, convergence, and performance of an adaptive control algorithm applied to a randomly varying system , 1989, Proceedings of the 28th IEEE Conference on Decision and Control,.

[26]  K. Chan,et al.  A note on the geometric ergodicity of a Markov chain , 1989, Advances in Applied Probability.

[27]  S. Meyn Ergodic theorems for discrete time stochastic systems using a stochastic lyapunov function , 1989 .

[28]  Sean P. Meyn,et al.  Adaptive Control of Time Varying Stochastic Systems , 1990 .

[29]  Sean P. Meyn,et al.  Asymptotic behavior of stochastic systems possessing Markovian realizations , 1991 .

[30]  S. Meyn,et al.  Stability of Markovian processes III: Foster–Lyapunov criteria for continuous-time processes , 1993, Advances in Applied Probability.

[31]  Sean P. Meyn,et al.  GEOMETRIC ERGODICITY OF A DOUBLY STOCHASTIC TIME SERIES MODEL , 1993 .

[32]  S. Meyn,et al.  Stability of Markovian processes II: continuous-time processes and sampled chains , 1993, Advances in Applied Probability.