Denumerable state continuous time Markov decision processes with unbounded cost and transition rates under average criterion

Abstract In this paper, we consider denumerable state continuous time Markov decision processes with (possibly unbounded) transition and cost rates under average criterion. We present a set of conditions and prove the existence of both average cost optimal stationary policies and a solution of the average optimality equation under the conditions. The results in this paper are applied to an admission control queue model and controlled birth and death processes.

[1]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[2]  E. Fainberg,et al.  On Homogeneous Markov Models with Continuous Time and Finite or Countable State Space , 1979 .

[3]  R. Serfozo.  Optimal control of random walks, birth and death processes, and queues , 1981, Advances in Applied Probability.

[4]  John Bather.  OPTIMAL STATIONARY POLICIES FOR DENUMERABLE MARKOV CHAINS IN CONTINUOUS TIME , 1976 .

[5]  J. Filar,et al.  Competitive Markov Decision Processes , 1996 .

[6]  I SennottLinn.  Average Cost Optimal Stationary Policies in Infinite State Markov Decision Processes with Unbounded Costs , 1989 .

[7]  郑少慧.  CONTINUOUS TIME MARKOV DECISION PROGRAMMING WITH AVERAGE REWARD CRITERION AND UNBOUNDED REWARD RATE , 1991 .

[8]  Linn I. Sennott,et al.  Average Cost Optimal Stationary Policies in Infinite State Markov Decision Processes with Unbounded Costs , 1989, Oper. Res..

[9]  Mark R. Lembersky.  On Maximal Rewards and $|varepsilon$-Optimal Policies in Continuous Time Markov Decision Chains , 1974 .

[10]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[11]  B. L. Miller.  Finite state continuous time Markov decision processes with an infinite planning horizon , 1968 .

[12]  Dimitri P. Bertsekas,et al.  Dynamic Programming: Deterministic and Stochastic Models , 1987 .

[13]  Kai Lai Chung,et al.  Markov Chains with Stationary Transition Probabilities , 1961 .

[14]  L. Sennott.  Another set of conditions for average optimality in Markov control processes , 1995 .

[15]  W. J. Anderson.  Continuous-Time Markov Chains , 1991 .

[16]  Zheng Shaohui.  Continuous time Markov decision programming with average reward criterion and unbounded reward rate , 1991 .

[17]  M. Puterman,et al.  Bias optimality in controlled queueing systems , 1998 .

[18]  C. Wu.  CONTINUOUS TIME MARKOV DECISION PROCESSES WITH UNBOUNDED REWARDS AND NON-UNIFORMLY BOUNDED TRANSITION RATES UNDER DISCOUNTED CRITERION , 1997 .

[19]  P. Kakumanu,et al.  Nondiscounted Continuous Time Markovian Decision Process with Countable State Space , 1972 .