A Markov decision process framework for optimal operation of monitored multi-state systems

We develop a decision support framework based on Markov decision processes to maximize the profit from the operation of a multi-state system. This framework enables a comprehensive management of the multi-state system, which considers the maintenance decisions together with those on the multi-state system operation setting, that is, its loading condition and configuration. The decisions are informed by a condition monitoring system, which estimates the health state of the multi-state system components. The approach is shown with reference to a mechanical system made up of components affected by fatigue.

[1]  Yishay Mansour,et al.  On the Complexity of Policy Iteration , 1999, UAI.

[2]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[3]  Yi Ding,et al.  Fuzzy universal generating functions for multi-state system reliability assessment , 2008, Fuzzy Sets Syst..

[4]  Amy J. C. Trappey,et al.  A multi-agent collaborative maintenance platform applying game theory negotiation strategies , 2013, J. Intell. Manuf..

[5]  K. Pipe.  Practical prognostics for Condition Based Maintenance , 2008, 2008 International Conference on Prognostics and Health Management.

[6]  Kevin E Lansey,et al.  Reliability'Availability Analysis of Municipal Water Distribution Networks: Case Studies , 2002 .

[7]  Nan Chen,et al.  New method for multi-state system reliability analysis based on linear algebraic representation , 2015 .

[8]  Branko Glisic,et al.  Value of information: impact of monitoring on decision‐making , 2014 .

[9]  Nagi Gebraeel,et al.  Stochastic modeling and real-time prognostics for multi-component systems with degradation rate interactions , 2014 .

[10]  Anatoly Lisnianski,et al.  Multi-state Markov Model for Reliability Analysis of a Combined Cycle Gas Turbine Power Plant , 2016, 2016 Second International Symposium on Stochastic Models in Reliability Engineering, Life Science and Operations Management (SMRLO).

[11]  Shu-Cherng Fang,et al.  Linear Optimization and Extensions: Theory and Algorithms , 1993 .

[12]  E. Zio,et al.  Prognostics and Health Management of Industrial Equipment , 2013 .

[13]  Enrico Zio,et al.  Evaluating maintenance policies by quantitative modeling and analysis , 2013, Reliab. Eng. Syst. Saf..

[14]  S. L. Phoenix,et al.  The Chain-of-Bundles Probability Model For the Strength of Fibrous Materials I: Analysis and Conjectures , 1978 .

[15]  Alessandro Giua,et al.  Guest Editorial , 2001, Discrete event dynamic systems.

[16]  Daming Lin,et al.  A review on machinery diagnostics and prognostics implementing condition-based maintenance , 2006 .

[17]  Gregory Levitin,et al.  Multi-State System Reliability - Assessment, Optimization and Applications , 2003, Series on Quality, Reliability and Engineering Statistics.

[18]  Antoine Grall,et al.  A condition-based maintenance policy for stochastically deteriorating systems , 2002, Reliab. Eng. Syst. Saf..

[19]  Leslie Pack Kaelbling,et al.  On the Complexity of Solving Markov Decision Problems , 1995, UAI.

[20]  Diederik J.D. Wijnmalen,et al.  Coordinated condition-based repair strategies for components of a multi-component maintenance system with discounts , 1997 .

[21]  Enrico Zio,et al.  Computational Methods for Reliability and Risk Analysis , 2009 .

[22]  Konstantinos Papakonstantinou,et al.  Optimum inspection and maintenance policies for corroded structures using partially observable Markov decision processes and stochastic, physically based models , 2014 .

[23]  David McMillan,et al.  Review of Markov models for maintenance optimization in the context of offshore wind , 2015 .

[24]  P. L’Ecuyer,et al.  A stochastic control approach to group preventive replacement in a multicomponent system , 1982 .

[25]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[26]  John Dalsgaard Sørensen,et al.  Methods for Risk-Based Planning of O&M of Wind Turbines , 2014 .

[27]  Milad Memarzadeh,et al.  Value of information in sequential decision making: Component inspection, permanent monitoring and system-level scheduling , 2016, Reliab. Eng. Syst. Saf..

[28]  Liu Hong,et al.  Review of interdependent infrastructure systems vulnerability analysis , 2011, 2011 2nd International Conference on Intelligent Control and Information Processing.

[29]  Enrico Zio,et al.  Monte Carlo-based filtering for fatigue crack growth estimation , 2009 .

[30]  Belle R. Upadhyaya,et al.  Lifecycle Prognostics: Transitioning between information types , 2015 .

[31]  Marco Wiering,et al.  Reinforcement Learning and Markov Decision Processes , 2012, Reinforcement Learning.

[32]  Antoine Grall,et al.  Modelling multicomponent systems to quantify reliability centred maintenance strategies , 2011 .

[33]  Nan Chen,et al.  A survey of condition-based maintenance modeling of multi-component systems , 2016, 2016 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM).

[34]  Alan Robins,et al.  Guest Editorial: Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science , 2010 .

[35]  E. Zio,et al.  Prognostics and Health Management of Industrial Equipment , 2019 .

[36]  G. K. Chan,et al.  Optimum maintenance policy with Markov processes , 2006 .

[37]  M. Shinozuka,et al.  Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part II: POMDP implementation , 2014, Reliab. Eng. Syst. Saf..

[38]  Ruud H. Teunter,et al.  Joint condition-based maintenance and inventory optimization for systems with multiple components , 2017, Eur. J. Oper. Res..

[39]  Wallace J. Hopp,et al.  An optimal structured policy for maintenance of partially observable aircraft engine components , 1998 .

[40]  M. Shinozuka,et al.  Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part I: Theory , 2014, Reliab. Eng. Syst. Saf..

[41]  Tiedo Tinga,et al.  The interplay between deployment and optimal maintenance intervals for complex multi-component systems , 2013 .

[42]  Wenjian Wang,et al.  A framework for evaluating the impact of structural health monitoring on bridge management , 2010 .