Enhancing and measuring the predictive capabilities of testing-effort dependent software reliability models

Software testing is necessary to accomplish highly reliable software systems. If the project manager can conduct well-planned testing activities, the consumption of related testing-resources will be cost-effective. Over the past 30 years, many software reliability growth models (SRGMs) have been proposed to estimate the reliability growth of software, and they are mostly applicable to the late stages of testing in software development. Thus far, it appears that most SRGMs do not take possible changes of testing-effort consumption rates into consideration. However, in some cases, the policies of testing-resource allocation could be changed or adjusted. Thus, in this paper, we will incorporate the important concept of multiple change-points into Weibull-type testing-effort functions. The applicability and performance of the proposed models are demonstrated through two real data sets. Experimental results show that the proposed models give a fairly accurate prediction capability. Finally, based on the proposed SRGM, constructive rules are developed for determining optimal software release times.

[1]  P. K. Kapur,et al.  A SOFTWARE RELIABILITY GROWTH MODEL WITH TESTING EFFORT DEPENDENT LEARNING FUNCTION FOR DISTRIBUTED SYSTEMS , 2004 .

[2]  John D. Musa,et al.  Software Reliability Engineering: More Reliable Software Faster and Cheaper , 2004 .

[3]  Amrit L. Goel,et al.  Optimum release time for software systems based on reliability and cost criteria , 1984, J. Syst. Softw..

[4]  Amrit L. Goel,et al.  Time-Dependent Error-Detection Rate Model for Software Reliability and Other Performance Measures , 1979, IEEE Transactions on Reliability.

[5]  Ping-Feng Pai,et al.  Software reliability forecasting by support vector machines with simulated annealing algorithms , 2006, J. Syst. Softw..

[6]  Tarak Shantilal Goradia,et al.  Dynamic Impact Analysis: Analyzing Error Propagation in Program Executions , 1993 .

[7]  Chin-Yu Huang,et al.  Performance analysis of software reliability growth models with testing-effort and change-point , 2005, J. Syst. Softw..

[8]  Yoshihiro Tohma,et al.  Hyper-geometric distribution model to estimate the number of residual software faults , 1989, [1989] Proceedings of the Thirteenth Annual International Computer Software & Applications Conference.

[9]  Hongwei Liu,et al.  Software reliability growth model with change-point and environmental function , 2006, J. Syst. Softw..

[10]  Huan-Jyh Shyur,et al.  A stochastic software reliability model with imperfect-debugging and change-point , 2003, J. Syst. Softw..

[11]  Chin-Yu Huang,et al.  Integrating generalized Weibull-type testing-effort function and multiple change-points into software reliability growth models , 2005, 12th Asia-Pacific Software Engineering Conference (APSEC'05).

[12]  Liang Tian,et al.  On-line prediction of software reliability using an evolutionary connectionist model , 2005, J. Syst. Softw..

[13]  Feng-Zhong Zou,et al.  A change‐point perspective on the software failure process , 2003, Softw. Test. Verification Reliab..

[14]  Michael R. Lyu.  Software Reliability Engineering: A Roadmap , 2007, Future of Software Engineering (FOSE '07).

[15]  Simon P. Wilson,et al.  Software Reliability Modeling , 1994 .

[16]  Xuemei Zhang,et al.  Some successful approaches to software reliability modeling in industry , 2005, J. Syst. Softw..

[17]  Hoang Pham,et al.  Software field failure rate prediction before software deployment , 2006, J. Syst. Softw..

[18]  A. Wood,et al.  Predicting Software Reliability , 1996, Computer.

[19]  Shigeru Yamada,et al.  Cost-Reliability Optimal Release Policies for Software Systems , 1985, IEEE Transactions on Reliability.

[20]  M. Xie,et al.  Software Reliability Models - Past, Present and Future , 2000 .

[21]  Giuliano Antoniol,et al.  Assessing staffing needs for a software maintenance project through queuing simulation , 2004, IEEE Transactions on Software Engineering.

[22]  Chin-Yu Huang,et al.  Analysis of incorporating logistic testing-effort function into software reliability modeling , 2002, IEEE Trans. Reliab..

[23]  Mitsuru Ohba,et al.  Does imperfect debugging affect software reliability growth? , 1989, ICSE '89.

[24]  C.-T. Lin,et al.  Software Reliability Analysis by Considering Fault Dependency and Debugging Time Lag , 2006, IEEE Transactions on Reliability.

[25]  Ming Zhao,et al.  On the log-power NHPP software reliability model , 1992, [1992] Proceedings Third International Symposium on Software Reliability Engineering.

[26]  Chenggang Bai,et al.  Bayesian network based software reliability prediction with an operational profile , 2005, J. Syst. Softw..

[27]  Michael R. Lyu,et al.  A Unified Scheme of Some Nonhomogenous Poisson Process Models for Software Reliability Estimation , 2003, IEEE Trans. Software Eng..

[28]  Karama Kanoun,et al.  A Method for Software Reliability Analysis and Prediction Application to the TROPICO-R Switching System , 1991, IEEE Trans. Software Eng..

[29]  Gregory Levitin,et al.  Robust recurrent neural network modeling for software fault detection and correction prediction , 2007, Reliab. Eng. Syst. Saf..

[30]  Shunji Osaki,et al.  Software-reliability growth with a Weibull test-effort: a model and application , 1993 .

[31]  Lawrence H. Putnam,et al.  A General Empirical Solution to the Macro Software Sizing and Estimating Problem , 1978, IEEE Transactions on Software Engineering.

[32]  Mohammad Ubaidullah Bokhari,et al.  Analysis of a software reliability growth models: the case of log-logistic test-effort function , 2006 .

[33]  Mary Shaw,et al.  Forecasting field defect rates using a combined time-based and metrics-based approach: a case study of OpenBSD , 2005, 16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05).

[34]  Ming Zhao.  Change-point problems in software and hardware reliability , 1993 .

[35]  Marvin V. Zelkowitz,et al.  Analyzing medium-scale software development , 1978, ICSE '78.

[36]  F. N. Parr,et al.  An Alternative to the Rayleigh Curve Model for Software Development Effort , 1980, IEEE Transactions on Software Engineering.

[37]  Yoshinobu Tamura,et al.  A flexible stochastic differential equation model in distributed development environment , 2006, Eur. J. Oper. Res..

[38]  M. Camuffo,et al.  Automatic software test generation , 1990 .

[39]  Philip J. Boland,et al.  A birth-process approach to Moranda's geometric software-reliability model , 2003, IEEE Trans. Reliab..

[40]  P. K. Kapur,et al.  Testing Effort Control using Flexible Software Reliability Growth Model with Change Point , 2006 .

[41]  Dai Pan,et al.  Architecture-based software reliability modeling , 2006, J. Syst. Softw..

[42]  Hoang Pham.  Software Reliability , 1999 .

[43]  Chin-Yu Huang,et al.  Optimal resource allocation for cost and reliability of modular software systems in the testing phase , 2006, J. Syst. Softw..

[44]  Suku Nair,et al.  A Model for Software Development Effort and Cost Estimation , 1997, IEEE Trans. Software Eng..

[45]  Roger S. Pressman,et al.  Software Engineering: A Practitioner's Approach , 1982 .

[46]  Yi-Ping Chang,et al.  ESTIMATION OF PARAMETERS FOR NONHOMOGENEOUS POISSON PROCESS: SOFTWARE RELIABILITY WITH CHANGE-POINT MODEL , 2001 .

[47]  Harvey S. Koch,et al.  Optimal Release Time of Computer Software , 1983, IEEE Transactions on Software Engineering.

[48]  John D. Musa,et al.  Software reliability - measurement, prediction, application , 1987, McGraw-Hill series in software engineering and technology.

[49]  Michael R. Lyu,et al.  Applying reliability models more effectively (software) , 1992, IEEE Software.