论文信息 - On the finite horizon Bellman equation for controlled Markov jump models with unbounded characteristics: existence and approximation

On the finite horizon Bellman equation for controlled Markov jump models with unbounded characteristics: existence and approximation

This paper studies the finite horizon Bellman equation for controlled Markov jump models with unbounded jump and cost rates. Under concrete growth conditions on the jump rates a method of time-discretization is used to: (i) prove the existence of a solution, (ii) construct a computationally attractive approximation scheme. The accuracy of this scheme is shown to be of linear order. An application to a controlled infinite server is included.

Nico M. van Dijk

[1] A. A. Yushkevich. Controlled Jump Markov Models , 1981 .

[2] Lawrence D. Stone,et al. Necessary and Sufficient Conditions for Optimal Control of Semi-Markov Jump Processes , 1973 .

[3] K. Jo. Continuous-time markov decision processes with nonzero terminal reward , 1984 .

[4] P. Varaiya,et al. Optimal Control of Jump Processes , 1977 .

[5] Arie Hordijk,et al. Discretization and Weak Convergence in Markov Decision Drift Processes , 1984, Math. Oper. Res..

[6] R. Rishel. Controls optimal from time t onward and dynamic programming for systems of controlled jump processes , 1976 .

[7] P. Kakumanu,et al. Relation between continuous and discrete time markovian decision problems , 1977 .

[8] N. Dijk. Controlled Markov processes : time discretization , 1984 .

[9] B. L. Miller. Finite State Continuous Time Markov Decision Processes with a Finite Planning Horizon , 1968 .

[10] I. Gihman,et al. Controlled Stochastic Processes , 1979 .

[11] B. L. Miller. Finite state continuous time Markov decision processes with an infinite planning horizon , 1968 .

[12] Stanley R. Pliska,et al. Controlled jump processes , 1975 .