Countable-state average-cost regenerative stopping problems

Abstract : Regenerative stopping problems are stopping problems which recommence from the initial state upon stopping. An algorithm is presented which solves a semi-Markov regenerative stopping problem with a finite number of continue actions by solving a sequence of stopping problems. New results for the optimal stopping problem are obtained as well as for the regenerative stopping problem. Two models in the literature are used as detailed examples of the algorithm. (Author)

[1]  Cyrus Derman,et al.  Replacement of periodically inspected equipment. (An optimal optional stopping rule) , 1960 .

[2]  William S. Jewell,et al.  Markov-Renewal Programming. I: Formulation, Finite Return Models , 1963 .

[3]  THE EXISTENCE OF PURE, STATIONARY OPTIMAL POLICIES FOR A CLASS OF MARKOV RENEWAL PROGRAMS, , 1966 .

[4]  E. Denardo.  CONTRACTION MAPPINGS IN THE THEORY UNDERLYING DYNAMIC PROGRAMMING , 1967 .

[5]  Bennett Fox,et al.  Existence of Stationary Optimal Policies for Some Markov Renewal Programs , 1967 .

[6]  Gerald J. Lieberman,et al.  A Markovian Decision Model for a Joint Replacement and Stocking Problem , 1967 .

[7]  R. Kaplan.  Optimal Investigation Strategies with Imperfect Information , 1969 .

[8]  S. Ross.  Average cost semi-markov decision processes , 1970, Journal of Applied Probability.

[9]  Colin E. Bell.  Improved Algorithms for Inventory and Replacement-Stocking Problems , 1970 .

[10]  Steven A. Lippman.  Maximal Average-Reward Policies for Semi-Markov Decision Processes With Arbitrary State and Action Space , 1971 .

[11]  E. Denardo.  Markov Renewal Programs with Small Interest Rates , 1971 .

[12]  David Siegmund,et al.  Great expectations: The theory of optimal stopping , 1971 .

[13]  Howard M. Taylor.  Optimal replacement under additive damage and other failure models , 1975 .

[14]  R. M. Feldman.  Optimal replacement with semi-Markov shock models , 1976 .

[15]  William P. Pierskalla,et al.  A survey of maintenance models: The control and surveillance of deteriorating systems , 1976 .

[16]  Optimality of the one step look-ahead stopping times , 1977 .

[17]  B. Bergman.  Optimal replacement under a general failure model , 1978, Advances in Applied Probability.

[18]  B. L. Miller,et al.  Optimal Investigation as a Regenerative Stopping Problem. , 1979 .

[19]  Martin L. Puterman,et al.  On the Convergence of Policy Iteration in Stationary Dynamic Programming , 1979, Math. Oper. Res..