On ergodicity and recurrence properties of a Markov chain by an application to an open jackson network

This paper gives an overview of recurrence and ergodicity properties of a Markov chain. Two new notions for ergodicity and recurrence are introduced. They are called μ -geometric ergodicity and μ -geometric recurrence respectively. The first condition generalises geometric as well as strong ergodicity. Our key theorem shows that μ -geometric ergodicity is equivalent to weak μ -geometric recurrence. The latter condition is verified for the time-discretised two-centre open Jackson network. Hence, the corresponding two-dimensional Markov chain is μ -geometrically and geometrically ergodic, but not strongly ergodic. A consequence of μ -geometric ergodicity with μ of product-form is the convergence of the Laplace-Stieltjes transforms of the marginal distributions. Consequently all moments converge.

[1]  F. G. Foster.  On the Stochastic Matrices Associated with Certain Queuing Processes , 1953 .

[2]  Kai Lai Chung,et al.  Markov Chains with Stationary Transition Probabilities , 1961 .

[3]  Kai Lai Chung,et al.  Markov Chains with Stationary Transition Probabilities , 1961 .

[4]  D. Vere-Jones.  GEOMETRIC ERGODICITY IN DENUMERABLE MARKOV CHAINS , 1962 .

[5]  J. Neveu,et al.  Mathematical foundations of the calculus of probability , 1965 .

[6]  Arie Hordijk,et al.  Dynamic programming and Markov potential theory , 1974 .

[7]  Ergodicity Using Mean Visit Times , 1976 .

[8]  Awi Federgruen,et al.  RECURRENCE CONDITIONS IN DENUMERABLE STATE MARKOV DECISION PROCESSES , 1977 .

[9]  R. Tweedie.  Criteria for classifying general Markov chains , 1976, Advances in Applied Probability.

[10]  A. Federgruen,et al.  A note on simultaneous recurrence conditions on a set of denumerable stochastic matrices : (preprint) , 1978 .

[11]  G. Luecke,et al.  Strongly ergodic Markov chains and rates of convergence using spectral conditions , 1978 .

[12]  R. Tweedie,et al.  Criteria for strong ergodicity of Markov chains , 1978, Journal of Applied Probability.

[13]  R. Tweedie,et al.  Geometric Ergodicity and R-positivity for General Markov Chains , 1978 .

[14]  Pekka Tuominen,et al.  Exponential decay and ergodicity of general Markov processes and their discrete skeletons , 1979, Advances in Applied Probability.

[15]  Dean Isaacson,et al.  A characterization of geometric ergodicity , 1979 .

[16]  R. Tweedie,et al.  Exponential ergodicity in Markovian queueing and dam models , 1979, Journal of Applied Probability.

[17]  Anthony Unwin,et al.  Reversibility and Stochastic Networks , 1980 .

[18]  R. L. Tweedie,et al.  Criteria for ergodicity, exponential ergodicity and strong ergodicity of Markov processes , 1981, Journal of Applied Probability.

[19]  B. Hajek.  Hitting-time and occupation-time bounds implied by drift analysis with applications , 1982, Advances in Applied Probability.

[20]  E. Nummelin,et al.  Geometric ergodicity of Harris recurrent Marcov chains with applications to renewal theory , 1982 .

[21]  R. Tweedie.  The existence of moments for stationary Markov chains , 1983, Journal of Applied Probability.

[22]  R. L. Tweedie,et al.  Probability, Statistics and Analysis: Criteria for rates of convergence of Markov chains, with application to queueing and storage theory , 1983 .

[23]  E. Nummelin.  General irreducible Markov chains and non-negative operators: List of symbols and notation , 1984 .

[24]  E. Nummelin.  General irreducible Markov chains and non-negative operators: Embedded renewal processes , 1984 .

[25]  H. Zijm.  THE OPTIMALITY EQUATIONS IN MULTICHAIN DENUMERABLE STATE MARKOV DECISION PROCESSES WITH THE AVERAGE COST CRITERION: THE BOUNDED COST CASE MULTISTAGE BAYESIAN ACCEPTANCE SAMPLING: OPTIMALITY OF A (z,c",c'^)-SAMPLING PLAN IN GASE OF A POLYA PRIOR DISTRIBUTION , 1985 .

[26]  Upendra Dave,et al.  Applied Probability and Queues , 1987 .

[27]  Wojciech Szpankowski,et al.  Stability Conditions for Multidimensional Queueing Systems with Computer Applications , 1988, Oper. Res..

[28]  Arie Hordijk,et al.  Average, Sensitive and Blackwell Optimal Policies in Denumerable Markov Decision Chains with Unbounded Rewards , 1988, Math. Oper. Res..

[29]  A. W. Kemp,et al.  Applied Probability and Queues , 1989 .

[30]  J. B. Lasserre,et al.  Conditions for Existence of Average and Blackwell Optimal Stationary Policies in Denumerable Markov Decision Processes , 1988 .

[31]  K. Chan,et al.  A note on the geometric ergodicity of a Markov chain , 1989, Advances in Applied Probability.

[32]  Wojciech Szpankowski,et al.  Towards computable stability criteria for some multidimensional stochastic processes , 1989 .

[33]  Flos Spieksma,et al.  The existence of sensitive optimal policies in two multi-dimensional queueing models , 1991 .

[34]  F. Spieksma,et al.  Geometric Ergodicity of the ALOHA-system and a Coupled Processors Model , 1991, Probability in the Engineering and Informational Sciences.

[35]  Rommert Dekker,et al.  Recurrence Conditions for Average and Blackwell Optimality in Denumerable State Markov Decision Chains , 1992, Math. Oper. Res..