Stability and ergodicity of piecewise deterministic Markov processes

The main goal of this paper is to establish some equivalence results on stability, recurrence between a piecewise deterministic Markov process (PDMP for short) {X(t)} and an embedded discrete-time Markov chain {¿n} generated by a Markov kernel G that can be explicitly characterized in terms of the three local characteristics of the PDMP contrary to the resolvent kernel. First we establish some important results characterizing {¿n} as a sampling of the PDMP {X(t)} and deriving a connection between the probability of the first return time to a set for the discrete-time Markov chains generated by G and the resolvent kernel R of the PDMP. From these results we obtain equivalence results regarding recurrence and positive recurrence between {X(t)} and {¿n}.

[1]  J. Azema,et al.  Mesure invariante sur les classes récurrentes des processus de Markov , 1967 .

[2]  R. Tweedie,et al.  The recurrence structure of general Markov processes , 1979, Advances in Applied Probability.

[3]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[4]  Oswaldo Luiz do Valle Costa.  Stationary distributions for piecewise-deterministic Markov processes , 1990 .

[5]  S. Meyn,et al.  Stability of Markovian processes I: criteria for discrete-time Chains , 1992, Advances in Applied Probability.

[6]  Sean P. Meyn,et al.  Generalized Resolvents and Harris Recurrence of Markov Processes , 1992 .

[7]  E. Nummelin.  General irreducible Markov chains and non-negative operators: List of symbols and notation , 1984 .

[8]  Mark H. Davis.  Markov Models and Optimization , 1995 .

[9]  Sean P. Meyn,et al.  State-Dependent Criteria for Convergence of Markov Chains , 1994 .

[10]  Sean P. Meyn,et al.  Stability and convergence of moments for multiclass queueing networks via fluid limit models , 1995, IEEE Trans. Autom. Control..

[11]  François Dufour,et al.  Stability and Ergodicity of Piecewise Deterministic Markov Processes , 2008 .

[12]  J. Azema,et al.  Propriétés relatives des processus de Markov récurrents , 1969 .

[13]  M. Dempster,et al.  Optimal capacity expansion under uncertainty , 1987, Advances in Applied Probability.

[14]  Esa Nummelin,et al.  On non-singular renewal kernels with an application to a semigroup of transition kernels , 1986 .

[15]  S. Meyn,et al.  Stability of Markovian processes II: continuous-time processes and sampled chains , 1993, Advances in Applied Probability.

[16]  S. Meyn,et al.  Stability of Markovian processes III: Foster–Lyapunov criteria for continuous-time processes , 1993, Advances in Applied Probability.

[17]  Oswaldo Luiz do Valle Costa,et al.  Stability of piecewise-deterministic Markov processes , 1999, Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304).

[18]  J. Dai.  On Positive Harris Recurrence of Multiclass Queueing Networks: A Unified Approach Via Fluid Limit Models , 1995 .

[19]  D. Vere-Jones.  Markov Chains , 1972, Nature.