A Comparative Analysis of Open Source Software Reliability

The purpose of this study is to compare the fitting (goodness-of-fit) and prediction capabilities of three reliability models using the failure data of five popular open source software (OSS) products. The failure data are modeled by Weibull and two other Non Homogenous Poisson Process (NHPP) models (Yamada S-Shaped and Schneidewind ). The OSS products considered are Eclipse , Apache HTTP Server 2 , Firefox , MPlayer OS X , and ClamWin Free Antivirus . Weibull is chosen due to its popularity in lifetime and its flexibility in modeling various distributions. On the other hand, among many software reliability models, the NHPP models are prevalent. The goodness-of-fit is based on the entire failure data collected. Prediction is accomplished by estimating the models parameters based on partial failure history and then applying the estimates to the entire time span for which failure data is collected. The outcomes show that a reliability model that fits the failure data well may not necessarily be a decent forecaster of future failure patterns.

[1]  M. F. Fuller,et al.  Practical Nonparametric Statistics; Nonparametric Statistical Inference , 1973 .

[2]  Swapna S. Gokhale,et al.  Reliability simulation of component-based software systems , 1998, Proceedings Ninth International Symposium on Software Reliability Engineering (Cat. No.98TB100257).

[3]  Sarah Brocklehurst,et al.  New Ways to Get Accurate Reliability Measures , 1992, IEEE Softw..

[4]  Norman F. Schneidewind,et al.  Analysis of error processes in computer software , 1975, Reliable Software.

[5]  Ye Wu,et al.  An architecture-based software reliability model , 1999, Proceedings 1999 Pacific Rim International Symposium on Dependable Computing.

[6]  Shigeru Yamada,et al.  S-Shaped Reliability Growth Modeling for Software Error Detection , 1983, IEEE Transactions on Reliability.

[7]  Norman F. Schneidewind,et al.  Applying reliability models to the space shuttle , 1992, IEEE Software.

[8]  Maurice H. Halstead,et al.  Elements of software science , 1977 .

[9]  J. Herbsleb,et al.  Two case studies of open source software development: Apache and Mozilla , 2002, TSEM.

[10]  James M. Bieman,et al.  Software reliability growth with test coverage , 2002, IEEE Trans. Reliab..

[11]  Eric S. Raymond,et al.  The cathedral and the bazaar - musings on Linux and Open Source by an accidental revolutionary , 2001 .

[12]  N. Schneidewind.  A Recommended Practice for Software Reliability , 2022 .

[13]  Martin L. Shooman,et al.  Probabilistic Models for Software reliability Prediction , 1972, Statistical Computer Performance Evaluation.

[14]  Bev Littlewood,et al.  A Bayesian Reliability Model with a Stochastically Monotone Failure Rate , 1974 .

[15]  K Okumoto,et al.  TIME-DEPENDENT ERROR-DETECTION RATE MODEL FOR SOFTWARE AND OTHER PERFORMANCE MEASURES , 1979 .

[16]  Tsutomu Ishida,et al.  Metrics and Models in Software Quality Engineering , 1995 .

[17]  Amrit L. Goel,et al.  Time-Dependent Error-Detection Rate Model for Software Reliability and Other Performance Measures , 1979, IEEE Transactions on Reliability.

[18]  R. A. Groeneveld,et al.  Practical Nonparametric Statistics (2nd ed). , 1981 .

[19]  Bev Littlewood,et al.  A Bayesian Reliability Growth Model for Computer Software , 1973 .

[20]  Hoang Pham,et al.  System Software Reliability , 1999 .

[21]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[22]  Israel Koren,et al.  Fault-Tolerant Systems , 2007 .

[23]  Hany H. Ammar,et al.  A scenario-based reliability analysis approach for component-based software , 2004, IEEE Transactions on Reliability.

[24]  Z. Jelinski,et al.  Software reliability Research , 1972, Statistical Computer Performance Evaluation.

[25]  Ying Zhou,et al.  Open source software reliability model , 2005, ACM SIGSOFT Softw. Eng. Notes.

[26]  Mitsuru Ohba,et al.  Software Reliability Analysis Models , 1984, IBM J. Res. Dev..

[27]  Kishor S. Trivedi.  Probability and Statistics with Reliability, Queuing, and Computer Science Applications , 1984 .

[28]  Bev Littlewood,et al.  Tools for the Analysis of the Accuracy of Software Reliability Predictions , 1986 .

[29]  A. P. Dawid,et al.  Present position and potential developments: some personal views , 1984 .

[30]  I. Secretary,et al.  IEEE Std 1633TM-2008, IEEE Recommended Practice on Software Reliability , 2009 .

[31]  Roger C. Cheung,et al.  A User-Oriented Software Reliability Model , 1978, IEEE Transactions on Software Engineering.