There is, so far, only limited practical experience applying solution schemes for real-life partially observable Markov decision processes (POMDP's). In this work we address the special-case POMDP associated with the famous machine-replacement problem. The machine deteriorates down a series of states according to known transition probabilities. A state is identified by a probability of producing a defective item. Only a sample of the produced items is observable at each stage, in which it is to be decided whether to replace the machine or not. We suggest a very simple heuristic decision-rule that can easily handle replacement-type problems of large size and which is based on the Howard solution of the fully observable version of the problem. By a simulation experimental design we compare the performance of this heuristic relative to the generic POMDP solution algorithm which has been proposed by Lovejoy.
[1]
C. White.
Optimal control-limit strategies for a partially observed replacement problem†
,
1979
.
[2]
Dimitri P. Bertsekas,et al.
Dynamic Programming and Stochastic Control
,
1977,
IEEE Transactions on Systems, Man, and Cybernetics.
[3]
Cyrus Derman,et al.
Finite State Markovian Decision Processes
,
1970
.
[4]
Zilla Sinuany-Stern,et al.
Replacement policy under partially observed Markov process
,
1993
.
[5]
Chelsea C. White,et al.
A survey of solution techniques for the partially observed Markov decision process
,
1991,
Ann. Oper. Res..
[6]
W. Lovejoy.
A survey of algorithmic methods for partially observed Markov decision processes
,
1991
.
[7]
Ellis L. Johnson.
Computation and Structure of Optimal Reset Policies
,
1967
.