Reliability analysis and optimal version-updating for open source software

Context: Although reliability is a major concern of most open source projects, research on this problem has attracted attention only recently. In addition, the optimal version-dating for open source software considering its special properties is not yet discussed. Objective: In this paper, the reliability analysis and optimal version-updating for open source software are studied. Method: A modified non-homogeneous Poisson process model is developed for open source software reliability modeling and analysis. Based on this model, optimal version-updating for open source software is investigated as well. In the decision process, the rapid release strategy and the level of reliability are the two most important factors. However, they are essentially contradicting with each other. In order to consider these two conflicting factors simultaneously, a new decision model based on multi-attribute utility theory is proposed. Results: Our models are tested on the real world data sets from two famous open source projects: Apache and GNOME. It is found that traditional software reliability models provide overestimations of the reliability of open source software. In addition, the proposed decision model can help management to make a rational decision on the optimal version-updating for open source software. Conclusion: Empirical results reveal that the proposed model for open source software reliability can describe the failure process more accurately. Furthermore, it can be seen that the proposed decision model can assist management to appropriately determine the optimal version-update time for open source software.

[1]  J. Herbsleb,et al.  Two case studies of open source software development: Apache and Mozilla , 2002, TSEM.

[2]  Mitsuru Ohba,et al.  Software Reliability Analysis Models , 1984, IBM J. Res. Dev..

[3]  Michael R. Lyu,et al.  Handbook of software reliability engineering , 1996 .

[4]  Min Xie,et al.  Software Reliability Modelling , 1991, Series on Quality, Reliability and Engineering Statistics.

[5]  Chin-Yu Huang,et al.  Analysis of incorporating logistic testing-effort function into software reliability modeling , 2002, IEEE Trans. Reliab..

[6]  Kris Ven,et al.  Challenges and strategies in the use of Open Source Software by Independent Software Vendors , 2008, Inf. Softw. Technol..

[7]  Yoshinobu Tamura,et al.  Optimisation analysis for reliability assessment based on stochastic differential equation modelling for open source software , 2009, Int. J. Syst. Sci..

[8]  Eric S. Raymond,et al.  The cathedral and the bazaar - musings on Linux and Open Source by an accidental revolutionary , 2001 .

[9]  Xiang Li,et al.  Sensitivity analysis of release time of software reliability models incorporating testing effort with multiple change-points , 2010 .

[10]  Alfred V. Aho,et al.  Do Crosscutting Concerns Cause Defects? , 2008, IEEE Transactions on Software Engineering.

[11]  Min Xie,et al.  A study of the sensitivity of software release time , 1998, J. Syst. Softw..

[12]  Adiel Teixeira de Almeida,et al.  Multi-attribute risk assessment for risk ranking of natural gas pipelines , 2009, Reliab. Eng. Syst. Saf..

[13]  Yoshinobu Tamura,et al.  A COMPONENT-ORIENTED RELIABILITY ASSESSMENT METHOD FOR OPEN SOURCE SOFTWARE , 2008 .

[14]  M.Ricafranca Romulo.,et al.  A STUDY OF UNCERTAINTY , 1970 .

[15]  Bo Yang,et al.  A Study of Uncertainty in Software Cost and Its Impact on Optimal Software Release Time , 2008, IEEE Transactions on Software Engineering.

[16]  Min Xie,et al.  A Study of the Effect of Imperfect Debugging on Software Development Cost , 2003, IEEE Trans. Software Eng..

[17]  Michael R. Lyu,et al.  An Assessment of Testing-Effort Dependent Software Reliability Growth Models , 2007, IEEE Transactions on Reliability.

[18]  Rudolf Ferenc,et al.  Using the Conceptual Cohesion of Classes for Fault Prediction in Object-Oriented Systems , 2008, IEEE Transactions on Software Engineering.

[19]  Barbara Paech,et al.  Exploring the relationship of a file's history and its fault-proneness: An empirical method and its application to open source programs , 2010, Inf. Softw. Technol..

[20]  Ioannis Stamelos,et al.  Survival analysis on the duration of open source projects , 2010, Inf. Softw. Technol..

[21]  P. Burger.  Embedded Case Study Methods: Integrating Quantitative and Qualitative Knowledge , 2001 .

[22]  A. Weale.  Embedded Case Study Methods: Integrating Quantitative and Qualitative Knowledge , 2003 .

[23]  Hoang Pham,et al.  Software reliability and cost models: Perspectives, comparison, and practice , 2003, Eur. J. Oper. Res..

[24]  Jouni Markkula,et al.  Assessing maintainability change over multiple software releases , 2008, J. Softw. Maintenance Res. Pract..

[25]  Ralph L. Keeney,et al.  Decisions with multiple objectives: preferences and value tradeoffs , 1976 .

[26]  W. Edwards,et al.  Decision Analysis and Behavioral Research , 1986 .

[27]  John D. Musa,et al.  Software reliability measurement , 1984, J. Syst. Softw..

[28]  Simon P. Wilson,et al.  Software Reliability Modeling , 1994 .

[29]  Chin-Yu Huang,et al.  Enhancing and measuring the predictive capabilities of testing-effort dependent software reliability models , 2008, J. Syst. Softw..

[30]  Yi Zhang,et al.  Classifying Software Changes: Clean or Buggy? , 2008, IEEE Transactions on Software Engineering.

[31]  R. L. Keeney,et al.  Decisions with Multiple Objectives: Preferences and Value Trade-Offs , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[32]  Shigeru Yamada,et al.  S-Shaped Reliability Growth Modeling for Software Error Detection , 1983, IEEE Transactions on Reliability.

[33]  Peter C. Fishburn,et al.  Utility theory for decision making , 1970 .

[34]  Tibor Gyimóthy,et al.  Empirical validation of object-oriented metrics on open source software for fault prediction , 2005, IEEE Transactions on Software Engineering.

[35]  Adiel Teixeira de Almeida,et al.  A multi-criteria decision model to determine inspection intervals of condition monitoring based on delay time analysis , 2009, Reliab. Eng. Syst. Saf..

[36]  Thong Ngee Goh,et al.  Adaptive ridge regression system for software cost estimating on multi-collinear datasets , 2010, J. Syst. Softw..

[37]  D HerbslebJames,et al.  Two case studies of open source software development , 2002 .

[38]  Michael R. Lyu,et al.  What is software reliability? , 1994, Proceedings of COMPASS'94 - 1994 IEEE 9th Annual Conference on Computer Assurance.

[39]  Hoang Pham,et al.  Comparisons of nonhomogeneous Poisson process software reliability models and its applications , 2000, Int. J. Syst. Sci..

[40]  John D. Musa,et al.  Software reliability - measurement, prediction, application , 1987, McGraw-Hill series in software engineering and technology.

[41]  Measurement , 2007 .

[42]  Amrit L. Goel,et al.  Time-Dependent Error-Detection Rate Model for Software Reliability and Other Performance Measures , 1979, IEEE Transactions on Reliability.

[43]  Yuan-Shun Dai,et al.  A model for availability analysis of distributed software/hardware systems , 2002, Inf. Softw. Technol..