A performance-centred approach to optimising maintenance of complex systems

Abstract This paper introduces performance-centred maintenance (PCM) as a novel approach to maintain systems when dual consideration is given to operational performance and degradation condition. We consider situations where performance and condition do not necessarily deteriorate at the same rate typified by, say, an ageing system still achieving good performance or a new system performing poorly. In this problem context, competing interests may arise between different decision-makers, such as operators and maintainers, since alternative strategies may benefit either performance or condition at the expense of the other. To address this challenge we introduce a theoretical framework for the PCM approach and discuss key characteristics of the modelling problem. The general PCM approach is motivated by a real-world industrial system for which maintenance decisions required to be optimised. A specific application is shown for the industry problem which we model by a Markov decision process capable of interrogating decisions over multiple time-scales. We obtain an exact solution using dynamic programming. We also explore a less computationally challenging heuristic using a reinforcement learning algorithm and evaluate its accuracy for the large-scale industry model. We show that optimal maintenance policies from a PCM model can provide decision support to both maintainers and operators taking account of both perspectives of the problem.

[1]  Ülkü Gürler,et al.  A maintenance policy for a system with multi-state components: an approximate solution , 2002, Reliab. Eng. Syst. Saf..

[2]  Tommi S. Jaakkola,et al.  Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms , 2000, Machine Learning.

[3]  Abhijit Gosavi,et al.  Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning , 2003 .

[4]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[5]  Jun Pei,et al.  Replacement and inventory control for a multi-customer product service system with decreasing replacement costs , 2019, Eur. J. Oper. Res..

[6]  Magdi Sami Moustafa,et al.  Optimal major and minimal maintenance policies for deteriorating systems , 2004, Reliab. Eng. Syst. Saf..

[7]  Rommert Dekker,et al.  Optimal maintenance of multi-component systems: a review , 2008 .

[8]  Matteo Cicciotti,et al.  Operational optimization of networks of compressors considering condition-based maintenance , 2016, Comput. Chem. Eng..

[9]  M. Shinozuka,et al.  Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part I: Theory , 2014, Reliab. Eng. Syst. Saf..

[10]  Phuc Do,et al.  Maintenance grouping strategy for multi-component systems with dynamic contexts , 2014, Reliab. Eng. Syst. Saf..

[11]  Dan M. Frangopol,et al.  Life-cycle of structural systems: recent achievements and future directions† , 2016 .

[12]  Yifan Zhou,et al.  Maintenance optimisation of a parallel-series system with stochastic and economic dependence under limited maintenance capacity , 2016, Reliab. Eng. Syst. Saf..

[13]  John Quigley,et al.  Failure and maintenance data extraction from power plant maintenance management databases , 2009 .

[14]  Tahir Ekin,et al.  Integrated maintenance and production planning with endogenous uncertain yield , 2017, Reliab. Eng. Syst. Saf..

[15]  Athena Zitrou,et al.  The signal model: A model for competing risks of opportunistic maintenance , 2011, Eur. J. Oper. Res..

[16]  Li Xia,et al.  A Structure Property of Optimal Policies for Maintenance Problems WithSafety-Critical Components , 2008, IEEE Transactions on Automation Science and Engineering.

[17]  Ling Li,et al.  Joint optimization of production, quality control and maintenance for serial-parallel multistage production systems , 2020, Reliab. Eng. Syst. Saf..

[18]  Ajith Kumar Parlikad,et al.  A review of asset management literature on multi-asset systems , 2019, Reliab. Eng. Syst. Saf..

[19]  Tim Bedford,et al.  Competing risks and opportunistic informative maintenance , 2009 .

[20]  Matthew Revie,et al.  Continuous-Observation Partially Observable Semi-Markov Decision Processes for Machine Maintenance , 2017, IEEE Transactions on Reliability.

[21]  Jun Wang,et al.  Joint optimization of condition-based maintenance and inventory control for a k-out-of-n: F system of multi-state degrading components , 2021, Eur. J. Oper. Res..

[22]  Tarkan Tan,et al.  Integrated maintenance and spare part optimization for moving assets , 2018 .

[23]  John Ringwood,et al.  Condition-based maintenance methods for marine renewable energy , 2016 .

[24]  Dan M. Frangopol,et al.  Life-cycle performance, management, and optimisation of structural systems under uncertainty: accomplishments and challenges 1 , 2011, Structures and Infrastructure Systems.

[25]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[26]  Ayse Sena Eruguz,et al.  Optimizing usage and maintenance decisions for k‐out‐of‐n systems of moving assets , 2017 .

[27]  Richard Curran,et al.  A practical dynamic programming based methodology for aircraft maintenance check scheduling optimization , 2020, Eur. J. Oper. Res..

[28]  Lin Ma,et al.  Maintenance optimisation of a series production system with intermediate buffers using a multi-agent FMDP , 2018, Reliab. Eng. Syst. Saf..

[29]  Diederik J.D. Wijnmalen,et al.  Optimum condition-based maintenance policies for deteriorating systems with partial information , 1996 .

[30]  Ruud H. Teunter,et al.  Joint condition-based maintenance and inventory optimization for systems with multiple components , 2017, Eur. J. Oper. Res..

[31]  Enrico Zio,et al.  A Markov decision process framework for optimal operation of monitored multi-state systems , 2018 .

[32]  Abhijit Gosavi,et al.  Reinforcement learning for long-run average cost , 2004, Eur. J. Oper. Res..

[33]  Donghua Zhou,et al.  Remaining useful life estimation - A review on the statistical data driven approaches , 2011, Eur. J. Oper. Res..

[34]  Mahmut Parlar,et al.  A survey of maintenance models for multi-unit systems , 1991 .

[35]  Geoff Kent,et al.  Condition-based inspection policies for boiler heat exchangers , 2020, Eur. J. Oper. Res..

[36]  Yu Liu,et al.  Dynamic selective maintenance optimization for multi-state systems over a finite horizon: A deep reinforcement learning approach , 2020, Eur. J. Oper. Res..

[37]  Sharareh Taghipour,et al.  Integrated production and maintenance scheduling for a single degrading machine with deterioration-based failures , 2020, Comput. Ind. Eng..

[38]  Ruud H. Teunter,et al.  Condition-based maintenance for systems with economic dependence and load sharing , 2018 .

[39]  Philip A. Scarf,et al.  A review on maintenance optimization , 2020, Eur. J. Oper. Res..

[40]  Jing Huang,et al.  Deep reinforcement learning based preventive maintenance policy for serial production lines , 2020, Expert Syst. Appl..

[41]  Jean-Pierre Kenné,et al.  Production and replacement policies for a deteriorating manufacturing system under random demand and quality , 2018, Eur. J. Oper. Res..

[42]  Ruud H. Teunter,et al.  Condition-based maintenance policies for systems with multiple dependent components: A review , 2017, Eur. J. Oper. Res..

[43]  Lin Wang,et al.  Joint production control and maintenance policy for a serial system with quality deterioration and stochastic demand , 2020, Reliab. Eng. Syst. Saf..

[44]  S. Mahadevan,et al.  Solving Semi-Markov Decision Problems Using Average Reward Reinforcement Learning , 1999 .

[45]  Jay Lee,et al.  Wind turbine performance degradation assessment based on a novel similarity metric for machine performance curves , 2016 .

[46]  Mark A. Shayman,et al.  Multitime scale Markov decision processes , 2003, IEEE Trans. Autom. Control..

[47]  Nan Chen,et al.  Optimal Inspection and Replacement Policies for Multi-Unit Systems Subject to Degradation , 2018, IEEE Transactions on Reliability.

[49]  Hongzhou Wang,et al.  A survey of maintenance policies of deteriorating systems , 2002, Eur. J. Oper. Res..

[50]  E. Zio,et al.  A reinforcement learning approach to optimal part flow management for gas turbine maintenance , 2020, Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability.

[51]  Armin Jabbarzadeh,et al.  A robust integrated production and preventive maintenance planning model for multi-state systems with uncertain demand and common cause failures , 2019, Journal of Manufacturing Systems.

[52]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[53]  J. Christopher Beck,et al.  Integrated maintenance planning and production scheduling with Markovian deteriorating machine conditions , 2014 .

[54]  Huiping Zhang,et al.  Research of an integrated decision model for production scheduling and maintenance planning with economic objective , 2019, Comput. Ind. Eng..

[55]  Heping Li,et al.  A condition-based maintenance policy for multi-component systems with Lévy copulas dependence , 2016, Reliab. Eng. Syst. Saf..

[56]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[57]  Hasan Hüseyin Erdem,et al.  Determination of performance degradation of a marine diesel engine by using curve based approach , 2016 .

[58]  Steven I. Marcus,et al.  Simulation-based Algorithms for Markov Decision Processes/ Hyeong Soo Chang ... [et al.] , 2013 .

[59]  Velusamy Subramaniam,et al.  Integrated control policy of production and preventive maintenance for a deteriorating manufacturing system , 2018, Comput. Ind. Eng..

[60]  Wujun Si,et al.  Deep reinforcement learning for condition-based maintenance planning of multi-component systems under dependent competing risks , 2020, Reliab. Eng. Syst. Saf..

[61]  Takayoshi Tamura,et al.  New approximate dynamic programming algorithms for large-scale undiscounted Markov decision processes and their application to optimize a production and distribution system , 2016, Eur. J. Oper. Res..

[62]  El Houssaine Aghezzaf,et al.  An integrated production and preventive maintenance planning model , 2007, Eur. J. Oper. Res..

[63]  Iwona Paprocka The model of maintenance planning and production scheduling for maximising robustness , 2019 .

[64]  Salih O. Duffuaa,et al.  Optimal joint maintenance and operation policies to maximise overall systems effectiveness , 2013 .

[65]  Michel Gendreau,et al.  Integrating production, maintenance and quality: A multi-period multi-product profit-maximization model , 2018, Reliab. Eng. Syst. Saf..

[66]  Abhijit Gosavi Variance-penalized Markov decision processes: dynamic programming and reinforcement learning techniques , 2014, Int. J. Gen. Syst..

[67]  Liudong Xing,et al.  Joint optimization of budget allocation and maintenance planning of multi-facility transportation infrastructure systems , 2021, Eur. J. Oper. Res..

[68]  Soumaya Yacout,et al.  Optimal preventive maintenance policy based on reinforcement learning of a fleet of military trucks , 2019, J. Intell. Manuf..