Interval Methods for Uncertain Markov Decision Processes

In this paper, interval methods for uncertain Markov decision processes are considered. That is, a controlled Markov set-chain model with a finite state is developed by an interval arithmetic analysis, and we will find Pareto optimal policies which maximize the discounted or average expected rewards over all stationary policies under some partial order. The optimal policies are characterized by a maximal solution of an optimality equation including efficient set function.

[1]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[2]  John G. Kemeny,et al.  Finite Markov Chains. , 1960 .

[3]  Rutherford Aris,et al.  Discrete Dynamic Programming , 1965, The Mathematical Gazette.

[4]  K. Hinderer,et al.  Foundations of Non-stationary Dynamic Programming with Discrete Time Parameter , 1970 .

[5]  J. Bather.  Optimal decision procedures for finite Markov chains. Part II: Communicating systems , 1973, Advances in Applied Probability.

[6]  J. Bather.  Optimal decision procedures for finite markov chains. Part I: Examples , 1973, Advances in Applied Probability.

[7]  Nagata Furukawa,et al.  Characterization of Optimal Policies in Vector-Valued Markovian Decision Processes , 1980, Math. Oper. Res..

[8]  D. White.  Multi-objective infinite-horizon discounted Markov decision processes , 1982 .

[9]  Yukio Takahashi,et al.  Weak D-Markov Chain and its Application to a Queueing Network , 1983, Computer Performance and Reliability.

[10]  A. Neumaier.  New techniques for the analysis of linear interval equations , 1984 .

[11]  Yukio Takahashi.  A Weak D-Markov Chain Approach to Tandem Queueing Networks , 1987, Computer Performance and Reliability.

[12]  D. J. Hartcfiel.  Cyclic markov set-chains , 1993 .

[13]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[14]  E. Seneta,et al.  On the theory of Markov set-chains , 1994, Advances in Applied Probability.

[15]  Masami Kurano,et al.  Controlled Markov Set-Chains with Set-valued rewards , 1998 .

[16]  Masanori Hosaka,et al.  CONTROLLED MARKOV SET-CHAINS WITH DISCOUNTING , 1998 .

[17]  Darald J. Hartfiel,et al.  Markov Set-Chains , 1998 .

[18]  Masanori Hosaka,et al.  NON-DISCOUNTED OPTIMAL POLICIES IN CONTROLLED MARKOV SET-CHAINS , 1999 .

[19]  Masanori Hosaka,et al.  Controlled Markov set-chains under average criteria , 2001, Appl. Math. Comput..