A simple suboptimal algorithm for system maintenanceunder partial observability

We suggest a heuristic solution procedure for Partially Observable Markov Decision Processes with finite action space and finite state space with infinite horizon. The algorithm is a fast, very simple general heuristic; it is applicable for multiple states (not necessarily ordered) multiple actions and various distribution functions. The quality of the algorithm is checked in this paper against existing analytical and empirical results for two specific models of machine replacement. One model refers to the case of two‐action and two‐system states with uniform observations (Grosfeld‐Nir [4]), and the other model refers to a case of many ordered states with binomial observations (Sinuany‐Stern et al. [11]). The paper also presents the model realization for various probability distribution functions applied to maintenance and quality control.

[1]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Stochastic Control , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[2]  D. J. White,et al.  Real Applications of Markov Decision Processes , 1985 .

[3]  Zilla Sinuany-Stern,et al.  PII: S0305-0548(96)00043-3 , 2002 .

[4]  Abraham Grosfeld-Nir,et al.  A Two-State Partially Observable Markov Decision Process with Uniformly Distributed Observations , 1996, Oper. Res..

[5]  D. J. White,et al.  Further Real Applications of Markov Decision Processes , 1988 .

[6]  G. B. Wetherill,et al.  Quality Control and Industrial Statistics , 1975 .

[7]  G. Monahan.  State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 1982 .

[8]  Daniel E. Lane,et al.  A Partially Observable Model of Decision Making by Fishermen , 1989, Oper. Res..

[9]  Zilla Sinuany-Stern,et al.  Replacement policy under partially observed Markov process , 1993 .

[10]  William S. Lovejoy,et al.  Computationally Feasible Bounds for Partially Observed Markov Decision Processes , 1991, Oper. Res..

[11]  Shinhong Kim,et al.  A Partially Observable Markov Decision Process with Lagged Information , 1987 .

[12]  S. Christian Albright,et al.  Structural Results for Partially Observable Markov Decision Processes , 1979, Oper. Res..

[13]  Chelsea C. White,et al.  A Markov Quality Control Process Subject to Partial Observation , 1977 .

[14]  Chelsea C. White,et al.  Finite-Memory Suboptimal Design for Partially Observed Markov Decision Processes , 1994, Oper. Res..

[15]  C. White.  Optimal control-limit strategies for a partially observed replacement problem† , 1979 .

[16]  C. White,et al.  Application of Jensen's inequality to adaptive suboptimal design , 1980 .