论文信息 - CONTROLLED MARKOV SET-CHAINS WITH DISCOUNTING

CONTROLLED MARKOV SET-CHAINS WITH DISCOUNTING

In the framework of discounted Markov decision processes, we consider the case that the transition probability varies in some given domain at each time and its variation is unknown or unobservable. To this end we introduce a new model, named controlled Markov set-chains, based on Markov set-chains, and discuss its optimization under some partial order. Also, a numerical example is given to explain the theoretical results and the computation.

[1] O. Hernández-Lerma. Adaptive Markov Control Processes , 1989 .

[2] D. Blackwell. Discrete Dynamic Programming , 1962 .

[3] Martin L. Puterman,et al. Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[4] A. Neumaier. New techniques for the analysis of linear interval equations , 1984 .

[5] D. J. Hartfiel. Component bounds on Markov set-chain limiting sets , 1991 .

[6] E. Seneta,et al. On the theory of Markov set-chains , 1994, Advances in Applied Probability.

[7] Ronald A. Howard,et al. Dynamic Programming and Markov Processes , 1960 .

[8] D. Hartfiel. On the limiting set of stochastic products $xA\sb{1}\cdots A\sb{k}$ , 1981 .

[9] D. J. Hartcfiel. Cyclic markov set-chains , 1993 .

[10] K. Hinderer,et al. Foundations of Non-stationary Dynamic Programming with Discrete Time Parameter , 1970 .

[11] Pravin Varaiya,et al. Stochastic Systems: Estimation, Identification, and Adaptive Control , 1986 .

[12] Stephen P. Brooks,et al. Markov Decision Processes. , 1989 .