Average Continuous Control of Piecewise Deterministic Markov Processes

In this paper we consider the long run average continuous control problem of piecewise-deterministic Markov processes (PDP's for short). The control variable acts on the jump rate λ and transition measure Q of the PDP. The main goal of the paper is to characterize the optimality equation of the problem in terms of integro-differential equations for the continuous-time problem as well as in terms of embedded discrete-time Markov chains associated to the PDP.

[1]  François Dufour,et al.  On the Poisson Equation for Piecewise-Deterministic Markov Processes , 2003, SIAM J. Control. Optim..

[2]  Jane J. Ye,et al.  Necessary and sufficient optimality conditions for control of piecewise deterministic markov processes , 1992 .

[3]  Jane J. Ye,et al.  Impulse Control of Piecewise Deterministic Markov Processes , 1995 .

[4]  A. A. Yushkevich,et al.  Bellman inequalities in markov decision deterministic drift processes , 1988 .

[5]  O. Hernández-Lerma,et al.  Discrete-time Markov control processes , 1999 .

[6]  O. Costa,et al.  Impulse and continuous control of piecewise deterministic Markov processes , 2000 .

[7]  S. Ethier,et al.  Markov Processes: Characterization and Convergence , 2005 .

[8]  Arie Hordijk,et al.  Markov Decision Drift Processes; Conditions for Optimality Obtained by Discretization , 1985, Math. Oper. Res..

[9]  O.L.V. Costa,et al.  Optimal stopping with continuous control of piecewise deterministic Markov processes , 2000 .

[10]  Mark H. A. Davis.  Piecewise‐Deterministic Markov Processes: A General Class of Non‐Diffusion Stochastic Models , 1984 .

[11]  D. Verms.  Optimal control of piecewise deterministic markov process , 1985 .

[12]  U. S. Gugerui.  Optimal stopping of a piecewise-deterministic markov process , 1986 .

[13]  Arie Hordijk,et al.  Discretization and Weak Convergence in Markov Decision Drift Processes , 1984, Math. Oper. Res..

[14]  Dariusz Gatarek,et al.  Optimality conditions for impulsive control of piecewise-deterministic processes , 1992, Math. Control. Signals Syst..

[15]  Juan Ye.  Optimal control of piecewise deterministic Markov processes. , 1990 .

[16]  A. A. Yushkevich,et al.  Continuous time markov decision processes with interventions , 1983 .

[17]  ANTHONY ALMUDEVAR,et al.  A Dynamic Programming Algorithm for the Optimal Control of Piecewise Deterministic Markov Processes , 2001, SIAM J. Control. Optim..

[18]  Oswaldo Luiz do Valle Costa,et al.  Stability of piecewise-deterministic Markov processes , 1999, Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304).