Stability of Markovian processes III: Foster–Lyapunov criteria for continuous-time processes

In Part I we developed stability concepts for discrete chains, together with Foster–Lyapunov criteria for them to hold. Part II was devoted to developing related stability concepts for continuous-time processes. In this paper we develop criteria for these forms of stability for continuous-parameter Markovian processes on general state spaces, based on Foster-Lyapunov inequalities for the extended generator. Such test function criteria are found for non-explosivity, non-evanescence, Harris recurrence, and positive Harris recurrence. These results are proved by systematic application of Dynkin's formula. We also strengthen known ergodic theorems, and especially exponential ergodic results, for continuous-time processes. In particular we are able to show that the test function approach provides a criterion for f-norm convergence, and bounding constants for such convergence in the exponential ergodic case. We apply the criteria to several specific processes, including linear stochastic systems under non-linear feedback, work-modulated queues, general release storage processes and risk processes.

[1]  J. Doob.  Stochastic processes , 1953 .

[2]  John Lamperti,et al.  Criteria for the recurrence or transience of stochastic process. I , 1960 .

[3]  W. Wonham.  Liapunov criteria for weak stochastic stability , 1966 .

[4]  J. Azema,et al.  Mesure invariante sur les classes récurrentes des processus de Markov , 1967 .

[5]  S. Foguel.  POSITIVE OPERATORS ON C(X) , 1969 .

[6]  E. Çinlar,et al.  On dams with additive inputs and a general release rule , 1972, Journal of Applied Probability.

[7]  R. Tweedie.  Sufficient conditions for ergodicity and recurrence of Markov chains on a general state space , 1975 .

[8]  R. Tweedie.  Sufficient conditions for regularity, recurrence and ergodicity of Markov processes , 1975, Mathematical Proceedings of the Cambridge Philosophical Society.

[9]  Certain Properties of Piecewise-Linear Markov Processes , 1976 .

[10]  J. Michael Harrison,et al.  The Stationary Distribution and First Exit Probabilities of a Storage Process with General Release Rule , 1976, Math. Oper. Res..

[11]  P. Brockwell.  Stationary distributions for dams with additive input and content-dependent release rate , 1977, Advances in Applied Probability.

[12]  R. Tweedie.  Criteria for classifying general Markov chains , 1976, Advances in Applied Probability.

[13]  J. Michael Harrison,et al.  The Recurrence Classification of Risk and Storage Processes , 1978, Math. Oper. Res..

[14]  Pekka Tuominen,et al.  Exponential decay and ergodicity of general Markov processes and their discrete skeletons , 1979, Advances in Applied Probability.

[15]  R. Tweedie,et al.  Exponential ergodicity in Markovian queueing and dam models , 1979, Journal of Applied Probability.

[16]  R. Getoor.  Transience and recurrence of Markov processes , 1980 .

[17]  R. Khasminskii.  Stochastic Stability of Differential Equations , 1980 .

[18]  R. L. Tweedie,et al.  Criteria for ergodicity, exponential ergodicity and strong ergodicity of Markov processes , 1981, Journal of Applied Probability.

[19]  E. Nummelin,et al.  Geometric ergodicity of Harris recurrent Marcov chains with applications to renewal theory , 1982 .

[20]  R. Tweedie,et al.  Storage processes with general release rule and additive inputs , 1982, Advances in Applied Probability.

[21]  E. Nummelin.  General irreducible Markov chains and non-negative operators: List of symbols and notation , 1984 .

[22]  Mark H. A. Davis.  Piecewise‐Deterministic Markov Processes: A General Class of Non‐Diffusion Stochastic Models , 1984 .

[23]  E. Nummelin.  General irreducible Markov chains and non-negative operators: Embedded renewal processes , 1984 .

[24]  L. Stettner.  On the Existence and Uniqueness of Invariant Measure for Continuous Time Markov Processes , 1986 .

[25]  W. Kliemann.  Recurrence and invariant measures for degenerate diffusions , 1987 .

[26]  R. Tweedie.  Invariant measures for Markov chains with no irreducibility assumptions , 1988 .

[27]  L. Rogers.  GENERAL THEORY OF MARKOV PROCESSES , 1989 .

[28]  Joanna Mitro.  General theory of markov processes , 1991 .

[29]  Mu-Fa Chen.  ON THREE CLASSICAL PROBLEMS FOR MARKOV CHAINS WITH CONTINUOUS TIME PARAMETERS , 1991 .

[30]  S. Meyn,et al.  Stability of Markovian processes I: criteria for discrete-time Chains , 1992, Advances in Applied Probability.

[31]  A. Hordijk,et al.  On ergodicity and recurrence properties of a Markov chain by an application to an open jackson network , 1992, Advances in Applied Probability.

[32]  Karl Sigman,et al.  WORK-MODULATED QUEUES WITH APPLICATIONS TO STORAGE PROCESSES , 1992 .

[33]  Sean P. Meyn,et al.  Generalized Resolvents and Harris Recurrence of Markov Processes , 1992 .

[34]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[35]  S. Meyn,et al.  Stability of Markovian processes II: continuous-time processes and sampled chains , 1993, Advances in Applied Probability.

[36]  Sean P. Meyn,et al.  Stability of Generalized Jackson Networks , 1994 .

[37]  Mark H. Davis.  Markov Models and Optimization , 1995 .