Average Optimality in Dynamic Programming with General State Space

A Markovian decision model with general state space, compact action space, and the average cost as criterion is considered. The existence of an optimal policy is shown via an optimality inequality in terms of the minimal average cost g and a relative value function w. The existence of some w is usually shown via relative compactness in a space of real-valued functions on the state space. Here it shall be shown that one can instead do with pointwise relative compactness in the set of real numbers if one makes use of a generalized lower limit of functions. An application to an inventory model is given.

[1]  Dariusz Gatarek,et al.  Optimality conditions for impulsive control of piecewise-deterministic processes , 1992, Math. Control. Signals Syst..

[2]  Jaroslav Kožešnk,et al.  Information Theory, Statistical Decision Functions, Random Processes , 1962 .

[3]  Arie Hordijk,et al.  A sufficient condition for the existence of an optimal policy with respect to the average cost criterion in markovian decision processes : Prepublication , 1971 .

[4]  A. Hordijk,et al.  On the optimality of ( s , S )-policies in continuous review inventory models , 1986 .

[5]  Manfred SchÄl,et al.  Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal , 1975 .

[6]  L. Sennott.  A new condition for the existence of optimal stationary policies in average cost Markov decision processes , 1986 .

[7]  Dariusz Gatarek,et al.  On the compactness method in general ergodic impulsive control of markov processes , 1990 .

[8]  Manfred Schäl,et al.  On the Second Optimality Equation for Semi-Markov Decision Models , 1992, Math. Oper. Res..

[9]  David Assaf,et al.  Invariant problems in dynamic programming- average reward criterion , 1980 .

[10]  Linn I. Sennott,et al.  Average Cost Semi-Markov Decision Processes and the Control of Queueing Systems , 1989, Probability in the Engineering and Informational Sciences.

[11]  Linn I. Sennott,et al.  Average Cost Optimal Stationary Policies in Infinite State Markov Decision Processes with Unbounded Costs , 1989, Oper. Res..

[12]  M. Schäl.  A selection theorem for optimization problems , 1974 .

[13]  Manfred Schäl,et al.  Stationary Policies in Dynamic Programming Models Under Compactness Assumptions , 1983, Math. Oper. Res..

[14]  S. Ross.  Arbitrary State Markovian Decision Processes , 1968 .

[15]  L. Brown,et al.  Measurable Selections of Extrema , 1973 .

[16]  J. P. Georgin,et al.  Estimation et controle des chaines de Markov sur des espaces arbitraires , 1978 .

[17]  H. M. Taylor.  Markovian sequential replacement processes , 1965 .