Optimal Interventions in Countable Jump Markov Processes

In this paper an intervention refers to an immediate change of the state of the system; between interventions, the continuous-time jump Markov process is uncontrollable, with "natural" jump intensities. The multicriteria control problem for such a model is considered, and the constrained version is investigated with the help of the Lagrange multipliers technique. All of the theory is illustrated by an example of the optimal control of epidemic with carriers.

[1]  A. Piunovskiy.  Optimal Control of Random Sequences in Problems with Constraints , 1997 .

[2]  E. Altman.  Constrained Markov Decision Processes , 1999 .

[3]  A. Hordijk,et al.  Constrained admission control to a queueing system , 1989, Advances in Applied Probability.

[4]  A. Yushkevich.  On Reducing a Jump Controllable Markov Model to a Model with Discrete Time , 1980 .

[5]  O. Hernández-Lerma,et al.  Further topics on discrete-time Markov control processes , 1999 .

[6]  Hongzhou Wang,et al.  A survey of maintenance policies of deteriorating systems , 2002, Eur. J. Oper. Res..

[7]  Alexey Piunovskiy.  Controlled jump Markov processes with interventions. , 2001 .

[8]  Xuerong Mao,et al.  Constrained Markovian decision processes: the dynamic programming approach , 2000, Oper. Res. Lett..

[9]  A. A. Yushkevich,et al.  Verification Theorems for Markov Decision Processes with Controlled Deterministic Drift and Gradual and Impulsive Controls , 1990 .

[10]  Eugene A. Feinberg,et al.  Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach , 2004, Math. Oper. Res..

[11]  M. K rn,et al.  Stochastic Optimal Control , 1988 .

[12]  A. Abakuks.  Optimal immunisation policies for epidemics , 1974, Advances in Applied Probability.

[13]  John Yuan,et al.  Optimal maintenance policy for a Markovian system under periodic inspection , 2001, Reliab. Eng. Syst. Saf..

[14]  Moshe Haviv,et al.  On constrained Markov decision processes , 1996, Oper. Res. Lett..

[15]  V. Rykov,et al.  Controlled Queueing Systems , 1995 .

[16]  A. B. Piunovskii.  A Controlled Jump Discounted Model with Constraints , 1998 .

[17]  O.L.V. Costa,et al.  Optimal stopping with continuous control of piecewise deterministic Markov processes , 2000 .