Estimation and Analysis of Some Generalized Multiple Change-Point Software Reliability Models

Software typically undergoes debugging during both a testing phase before product release, and an operational phase after product release. But it is noted that the fault detection and removal processes during software development and operation are different. For example, the fault removal during operation occurs generally at a slower pace than development. In this paper, we derive a powerful, easily deployable technique for software reliability prediction and assessment in the testing and operational phases. We first review how several existing software reliability growth models (SRGM) based on non- homogeneous Poisson processes (NHPP) can be readily derived from a unified theory. With the unified theory, we further incorporate the concept of multiple change-points, i.e. points in time when the software environment changes, into software reliability modeling. Several models are proposed and discussed under both ideal and imperfect debugging conditions. We estimate the parameters of the proposed models by employing real software failure data, and give a fair comparison with some existing SRGM. Numerical results show that the proposed models can provide good software reliability prediction in the various stages of software development and operation. Our approach is flexible; we can model various environments ranging from exponential-type to S-shaped NHPP models.

[1]  Kishor S. Trivedi,et al.  A multiphase software reliability model: From testing to operational phase , 1996 .

[2]  Chin-Yu Huang,et al.  Analysis of Software Reliability Modeling Considering Testing Compression Factor and Failure-to-Fault Relationship , 2010, IEEE Transactions on Computers.

[3]  H. E. Dunsmore,et al.  Software engineering metrics and models , 1986 .

[4]  Shigeru Yamada,et al.  MARKOVIAN AVAILABILITY MEASUREMENT WITH TWO TYPES OF SOFTWARE FAILURES DURING THE OPERATION PHASE , 1999 .

[5]  John D. Musa,et al.  Software reliability engineering : more reliable software, faster development and testing , 1999 .

[6]  Swapna S. Gokhale,et al.  A simulation approach to structure-based software reliability analysis , 2005, IEEE Transactions on Software Engineering.

[7]  Min Xie,et al.  A Study of the Effect of Imperfect Debugging on Software Development Cost , 2003, IEEE Trans. Software Eng..

[8]  M. Xie,et al.  Software Reliability Models - Past, Present and Future , 2000 .

[9]  Min Xie,et al.  Software Reliability Modelling , 1991, Series on Quality, Reliability and Engineering Statistics.

[10]  Michael R. Lyu,et al.  Optimal testing resource allocation, and sensitivity analysis in software development , 2005, IEEE Transactions on Reliability.

[11]  Amrit L. Goel,et al.  An Analysis Of Recurrent Software Errors In A Real-Time Control System , 1978, ACM Annual Conference.

[12]  Jinde Wang,et al.  Parameter Estimation of Some NHPP Software Reliability Models with Change-Point , 2005 .

[13]  Yoshinobu Tamura,et al.  Comparison of Software Reliability Assessment Methods for Open Source Software , 2005, 11th International Conference on Parallel and Distributed Systems (ICPADS'05).

[14]  Upendra Dave,et al.  Statistical Analysis of Software Reliability Models , 1995 .

[15]  Richard F. Gunst,et al.  Applied Regression Analysis , 1999, Technometrics.

[16]  A. L. Goel Relating operational software reliability and workload: results from an experimental study , 1996, Proceedings of 1996 Annual Reliability and Maintainability Symposium.

[17]  Hoang Pham,et al.  A general imperfect-software-debugging model with S-shaped fault-detection rate , 1999 .

[18]  B. Littlewood,et al.  Dependability assessment of software-based systems: state of the art , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[19]  Paul Ammann,et al.  The Effect of Imperfect Error Detection on Reliability Assessment via Life Testing , 1992, IEEE Trans. Software Eng..

[20]  Ravishankar K. Iyer,et al.  Field Failures And Reliability In Operation , 1993, Proceedings of 1993 IEEE International Symposium on Software Reliability Engineering.

[21]  S. Yamada,et al.  Optimum software-release time considering an error-detection phenomenon during operation , 1990 .

[22]  Karama Kanoun,et al.  A Method for Software Reliability Analysis and Prediction Application to the TROPICO-R Switching System , 1991, IEEE Trans. Software Eng..

[23]  Chin-Yu Huang,et al.  Staffing Level and Cost Analyses for Software Debugging Activities Through Rate-Based Simulation Approaches , 2009, IEEE Transactions on Reliability.

[24]  Michael R. Lyu,et al.  Quantitative software reliability modeling from testing to operation , 2000, Proceedings 11th International Symposium on Software Reliability Engineering. ISSRE 2000.

[25]  Chin-Yu Huang,et al.  Reliability prediction and assessment of fielded software based on multiple change-point models , 2005, 11th Pacific Rim International Symposium on Dependable Computing (PRDC'05).

[26]  W. Hays Applied Regression Analysis. 2nd ed. , 1981 .

[27]  Chin-Yu Huang,et al.  Enhancing and measuring the predictive capabilities of testing-effort dependent software reliability models , 2008, J. Syst. Softw..

[28]  Shih-Wei Huang,et al.  CARATS: A Computer-Aided Reliability Assessment Tool for Software Based on Object-Oriented Design , 2006, TENCON 2006 - 2006 IEEE Region 10 Conference.

[29]  Ming Zhao,et al.  Statistical Reliability Change-point Estimation Models , 2003 .

[30]  Michael R. Lyu Software Reliability Engineering: A Roadmap , 2007, Future of Software Engineering (FOSE '07).

[31]  Paul Rook,et al.  Software Reliability Handbook , 1990 .

[32]  John D. Musa,et al.  Software reliability - measurement, prediction, application , 1987, McGraw-Hill series in software engineering and technology.

[33]  Yinong Chen Modelling software operational reliability via input domain-based reliability growth model , 1998, Digest of Papers. Twenty-Eighth Annual International Symposium on Fault-Tolerant Computing (Cat. No.98CB36224).

[34]  Chin-Yu Huang,et al.  Software Reliability Analysis and Measurement Using Finite and Infinite Server Queueing Models , 2008, IEEE Transactions on Reliability.

[35]  Simon P. Wilson,et al.  Software Reliability Modeling , 1994 .

[36]  Michael R. Lyu,et al.  Handbook of software reliability engineering , 1996 .

[37]  S. J. Keene,et al.  Reliability growth of fielded software , 1993, Annual Reliability and Maintainability Symposium 1993 Proceedings.

[38]  Michael R. Lyu,et al.  A Unified Scheme of Some Nonhomogenous Poisson Process Models for Software Reliability Estimation , 2003, IEEE Trans. Software Eng..

[39]  Xiaolin Teng,et al.  Considering fault removal efficiency in software reliability assessment , 2003, IEEE Trans. Syst. Man Cybern. Part A.

[40]  G. Q. Kenny Estimating defects in commercial software during operational use , 1993 .

[41]  Shigeru Yamada,et al.  Software reliability measurement in imperfect debugging environment and its application , 1993 .

[42]  Sarah Brocklehurst,et al.  Recalibrating Software Reliability Models , 1990, IEEE Trans. Software Eng..

[43]  N. Draper,et al.  Applied Regression Analysis: Draper/Applied Regression Analysis , 1998 .

[44]  Xiang Li,et al.  Sensitivity analysis of release time of software reliability models incorporating testing effort with multiple change-points , 2010 .

[45]  Hongwei Liu,et al.  Software reliability growth model with change-point and environmental function , 2006, J. Syst. Softw..

[46]  Chin-Yu Huang,et al.  Performance analysis of software reliability growth models with testing-effort and change-point , 2005, J. Syst. Softw..

[47]  Chin-Yu Huang,et al.  Reliability analysis using weighted combinational models for web-based software , 2009, WWW '09.

[48]  Min Xie,et al.  A study of operational and testing reliability in software reliability analysis , 2000, Reliab. Eng. Syst. Saf..

[49]  Willa K. Ehrlich,et al.  Modeling software failures and reliability growth during system testing , 1987, ICSE '87.

[50]  Hiroyuki Okamura,et al.  A reliability assessment method for software products in operational phase—proposal of an accelerated life testing model , 2001 .

[51]  Xuemei Zhang,et al.  Adjusting software failure rates that are estimated from test data , 2005, IEEE Transactions on Reliability.

[52]  Mitsuru Ohba,et al.  Does imperfect debugging affect software reliability growth? , 1989, ICSE '89.

[53]  William A. Florac,et al.  Measuring the Software Process: Statistical Process Control for Software Process Improvement , 1999 .