Approximation and bounds in discrete event dynamic programming

This paper presents a general dynamic programming algorithm for the solution of optimal stochastic control problems concerning a class of discrete event systems. The emphasis is put on the numerical technique used for the approximation of the solution of the dynamic programming equation. This approach can be efficiently used for the solution of optimal control problems concerning Markov renewal processes. This is illustrated on a group preventive replacement model generalizing an earlier work of the authors.

[1]  J. MacQueen.  A MODIFIED DYNAMIC PROGRAMMING METHOD FOR MARKOVIAN DECISION PROBLEMS , 1966 .

[2]  E. Denardo.  CONTRACTION MAPPINGS IN THE THEORY UNDERLYING DYNAMIC PROGRAMMING , 1967 .

[3]  J. MacQueen,et al.  Letter to the Editor - A Test for Suboptimal Actions in Markovian Decision Problems , 1967, Oper. Res..

[4]  Evan L. Porteus.  Some Bounds for Discounted Sequential Decision Processes , 1971 .

[5]  B. Fox.  Discretizing dynamic programs , 1973 .

[6]  Evan L. Porteus.  Bounds and Transformations for Discounted Finite Markov Decision Chains , 1975, Oper. Res..

[7]  Evan L. Porteus.  On the Optimality of Structured Policies in Countable Stage Decision Processes , 1975 .

[8]  Manfred SchÄl,et al.  Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal , 1975 .

[9]  D. Bertsekas.  Convergence of discretization procedures in dynamic programming , 1975 .

[10]  James W. Daniel,et al.  Splines and efficiency in dynamic programming , 1976 .

[11]  Ward Whitt,et al.  Approximations of Dynamic Programs, I , 1978, Math. Oper. Res..

[12]  Thomas L. Morin,et al.  COMPUTATIONAL ADVANCES IN DYNAMIC PROGRAMMING , 1978 .

[13]  K. Hinderer.  ON APPROXIMATE SOLUTIONS OF FINITE-STAGE DYNAMIC PROGRAMS , 1978 .

[14]  P. Whittle.  A simple condition for regularity in negative programming , 1979, Journal of Applied Probability.

[15]  A. Alj,et al.  Hierarchical Control of a Population Process with Application to Group Preventive Maintenance , 1980 .

[16]  Raymond Rishel,et al.  Group preventive maintenance An example of controlled jump processes , 1981, 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[17]  Hans-Joachim Langen,et al.  Convergence of Dynamic Programming Models , 1981, Math. Oper. Res..

[18]  Martin L. Puterman,et al.  Action Elimination Procedures for Modified Policy Iteration Algorithms , 1982, Oper. Res..

[19]  P. L’Ecuyer,et al.  A stochastic control approach to group preventive replacement in a multicomponent system , 1982 .

[20]  L. Thomas,et al.  Computational comparison of value iteration algorithms for discounted Markov decision processes , 1983 .