Value iteration and rolling plans for Markov control processes with unbounded rewards

The authors study the convergence of value-iteration functions and the existence of error bounds for rolling horizon procedures in discrete-time Markov control processes with Borel state and control spaces, and unbounded reward functions with a discount factor. As expected, in contrast to the bounded case, the bounds are 'pointwise,' not 'uniform'. In addition, it is shown how the error bound in the weighted norm case can be improved by introducing appropriate ergodicity conditions.<<ETX>>

[1]  J. Wessels.  Markov programming by successive approximations by respect to weighted supremum norms , 1976, Advances in Applied Probability.

[2]  S. Lippman.  On Dynamic Programming with Unbounded Rewards , 1975 .

[3]  Manfred SchÄl,et al.  Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal , 1975 .

[4]  J Jaap Wessels,et al.  Note---A Note on Dynamic Programming with Unbounded Rewards , 1975 .

[5]  T. Parthasarathy,et al.  Optimal Plans for Dynamic Programming Problems , 1976, Math. Oper. Res..

[6]  Daniel F. Spulber,et al.  Stochastic Equilibrium and Optimality with Rolling Plans , 1981 .

[7]  Alain Bensoussan.  Stochastic control in discrete time and applications to the theory of production , 1982 .

[8]  R. Cavazos-Cadena.  Finite-state approximations for denumerable state discounted markov decision processes , 1986 .

[9]  K. Wakuta,et al.  Arbitrary state semi-Markov decision processes , 1987 .

[10]  Schäl Manfred.  Estimation and control in discounted stochastic dynamic programming , 1987 .

[11]  Mukul Majumdar,et al.  Controlled semi-Markov models, the discounted case , 1989 .

[12]  S. Meyn.  Ergodic theorems for discrete time stochastic systems using a stochastic lyapunov function , 1989 .

[13]  O. Hernández-Lerma,et al.  Error bounds for rolling horizon policies in discrete-time Markov control processes , 1990 .

[14]  Raymond L. Smith,et al.  Rolling Horizon Procedures in Nonhomogeneous Markov Decision Processes , 1992, Oper. Res..