Simulation-Based Optimization Approach for Software Cost Model with Rejuvenation

Software rejuvenation is a preventive and proactive maintenance solution that is particularly useful for counteracting the phenomenon of software aging. In this paper we consider an operational software system with multiple degradations and derive the optimal software rejuvenation policy minimizing the expected operation cost per unit time in the steady state, via the dynamic programing approach. Especially, we develop a reinforcement learning algorithm to estimate the optimal rejuvenation schedule adaptively and examine its asymptotic properties through a simulation experiment.

[1]  Edward N. Adams,et al.  Optimizing Preventive Service of Software Products , 1984, IBM J. Res. Dev..

[2]  Tadashi Dohi,et al.  Estimating Software Rejuvenation Schedules in High-Assurance Systems , 2001, Comput. J..

[3]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[4]  Vivek S. Borkar,et al.  Actor-Critic - Type Learning Algorithms for Markov Decision Processes , 1999, SIAM J. Control. Optim..

[5]  Abhijit Gosavi,et al.  Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning , 2003 .

[6]  Abhijit Gosavi,et al.  Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning , 2003 .

[7]  Vivek S. Borkar,et al.  Learning Algorithms for Markov Decision Processes with Average Cost , 2001, SIAM J. Control. Optim..

[8]  Kishor S. Trivedi,et al.  Optimal rejuvenation for tolerating soft failures , 1996 .

[9]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[10]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[11]  Matteo Sereno,et al.  Fine Grained Software Degradation Models for Optimal Rejuvenation Policies , 2001, Perform. Evaluation.

[12]  Sean P. Meyn,et al.  The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning , 2000, SIAM J. Control. Optim..

[13]  Richard S. Sutton,et al.  Reinforcement Learning , 1992, Handbook of Machine Learning.

[14]  Elaine J. Weyuker,et al.  Monitoring Smoothly Degrading Systems for Increased Dependability , 2004, Empirical Software Engineering.

[15]  Hoang Pham,et al.  Handbook of reliability engineering , 2013 .

[16]  W. D. Ray,et al.  Stochastic Models: An Algorithmic Approach , 1995 .

[17]  Kishor S. Trivedi,et al.  Proactive management of software aging , 2001, IBM J. Res. Dev..

[18]  Yennun Huang,et al.  Software rejuvenation: analysis, module and applications , 1995, Twenty-Fifth International Symposium on Fault-Tolerant Computing. Digest of Papers.

[19]  Kishor S. Trivedi,et al.  Optimal Software Rejuvenation for Tolerating Soft Failures , 1996, Perform. Evaluation.

[20]  Kishor S. Trivedi,et al.  Analysis of Preventive Maintenance in Transactions Based Software Systems , 1998, IEEE Trans. Computers.

[21]  Tadashi Dohi,et al.  Software Rejuvenation: Modeling and Applications , 2003 .

[22]  Kishor S. Trivedi,et al.  Analysis of software rejuvenation using Markov Regenerative Stochastic Petri Net , 1995, Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'95.

[23]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .