Defining and Evaluating a Measure of Open Source Project Survivability

In this paper, we define and validate a new multidimensional measure of Open Source Software (OSS) project survivability, called Project Viability. Project viability has three dimensions: vigor, resilience, and organization. We define each of these dimensions and formulate an index called the Viability Index (VI) to combine all three dimensions. Archival data of projects hosted at SourceForge.net are used for the empirical validation of the measure. An Analysis Sample (n=136) is used to assign weights to each dimension of project viability and to determine a suitable cut-off point for VI. Cross-validation of the measure is performed on a hold-out Validation Sample (n=96). We demonstrate that project viability is a robust and valid measure of OSS project survivability that can be used to predict the failure or survival of an OSS project accurately. It is a tangible measure that can be used by organizations to compare various OSS projects and to make informed decisions regarding investment in the OSS domain.

[1]  Kurt R. Linberg.  Software developer perceptions about software project failure: a case study , 1999, J. Syst. Softw..

[2]  E. DeLong,et al.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. , 1988, Biometrics.

[3]  Wei Li,et al.  Object-Oriented Metrics Which Predict Maintainability , 1993 .

[4]  Victor R. Basili,et al.  A Validation of Object-Oriented Design Metrics as Quality Indicators , 1996, IEEE Trans. Software Eng..

[5]  Robert Costanza,et al.  What is a healthy ecosystem? , 1999, Aquatic Ecology.

[6]  Stefan Koch,et al.  Software evolution in open source projects - a large-scale investigation , 2007, J. Softw. Maintenance Res. Pract..

[7]  Norman E. Fenton,et al.  Software Metrics: A Rigorous Approach , 1991 .

[8]  Taghi M. Khoshgoftaar,et al.  Application of a usage profile in software quality models , 1999, Proceedings of the Third European Conference on Software Maintenance and Reengineering (Cat. No. PR00090).

[9]  N. Perkins,et al.  Optimal Cut-point and Its Corresponding Youden Index to Discriminate Individuals Using Pooled Blood Samples , 2005, Epidemiology.

[10]  Kris Ven,et al.  Should You Adopt Open Source Software? , 2008, IEEE Software.

[11]  Daniel German,et al.  Mining CVS repositories, the softChange experience , 2004, MSR.

[12]  Tibor Gyimóthy,et al.  Empirical validation of object-oriented metrics on open source software for fault prediction , 2005, IEEE Transactions on Software Engineering.

[13]  Mohammad Alshayeb,et al.  An Empirical Validation of Object-Oriented Metrics in Two Different Iterative Software Processes , 2003, IEEE Trans. Software Eng..

[14]  Barry W. Boehm,et al.  Improving Software Productivity , 1987, Computer.

[15]  Sandro Morasca,et al.  Defining and Validating Measures for Object-Based High-Level Design , 1999, IEEE Trans. Software Eng..

[16]  S. Menard.  Applied Logistic Regression Analysis , 1996 .

[17]  Victor R. Basili,et al.  Editorial: Open Source and Empirical Software Engineering , 2001, Empirical Software Engineering.

[18]  Victor R. Basili,et al.  Understanding and predicting the process of software maintenance releases , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[19]  H. E. Dunsmore,et al.  Software engineering metrics and models , 1986 .

[20]  Sandeep Krishnamurthy,et al.  Cave or Community? An Empirical Examination of 100 Mature Open Source Projects , 2002, First Monday.

[21]  Shari Lawrence Pfleeger,et al.  Preliminary Guidelines for Empirical Research in Software Engineering , 2002, IEEE Trans. Software Eng..

[22]  John C. Munson,et al.  An approach to the measurement of software evolution: Research Articles , 2005 .

[23]  Giancarlo Succi,et al.  An empirical study of open-source and closed-source software products , 2004, IEEE Transactions on Software Engineering.

[24]  John C. Munson,et al.  An approach to the measurement of software evolution , 2005, J. Softw. Maintenance Res. Pract..

[25]  Sallie M. Henry,et al.  Object-oriented metrics that predict maintainability , 1993, J. Syst. Softw..

[26]  Shari Lawrence Pfleeger,et al.  Towards a Framework for Software Measurement Validation , 1995, IEEE Trans. Software Eng..

[27]  P. Allison.  Measures of Inequality , 1978 .

[28]  Yuanyuan Zhou,et al.  Rx: treating bugs as allergies---a safe method to survive software failures , 2005, SOSP '05.

[29]  Neville Churcher,et al.  Comments on "A Metrics Suite for Object Oriented Design" , 1995, IEEE Trans. Software Eng..

[30]  Nozer D. Singpurwalla,et al.  To survive or to fail: That is the question , 1994 .

[31]  Lionel C. Briand,et al.  Assessing, Comparing, and Combining Statechart- based testing and Structural testing: An Experiment , 2007, ESEM 2007.

[32]  Ramanath Subramanyam,et al.  Empirical Analysis of CK Metrics for Object-Oriented Design Complexity: Implications for Software Defects , 2003, IEEE Trans. Software Eng..

[33]  Kieran Healy,et al.  The Ecology of Open-Source Software Development , 2003 .

[34]  Kevin Crowston,et al.  Self-organization of teams for free/libre open source software development , 2007, Inf. Softw. Technol..

[35]  Sandro Morasca,et al.  On the application of measurement theory in software engineering , 2004, Empirical Software Engineering.

[36]  Rudolph E. Seviora,et al.  Aspect-oriented implementation of software health indicators , 2001, Proceedings Eighth Asia-Pacific Software Engineering Conference.

[37]  Albert L. Baker,et al.  A mathematical perspective for software measures research , 1990, Softw. Eng. J..

[38]  Gary L. Lilien,et al.  Location, Location, Location: How Network Embeddedness Affects Project Success in Open Source Systems , 2006, Manag. Sci..

[39]  George J. Vachtsevanos,et al.  Application Challenges: System Health Management for Complex Systems , 2000, IPDPS Workshops.

[40]  Kevin Crowston,et al.  Assessing the health of open source communities , 2006, Computer.

[41]  Anna Sidorova,et al.  SURVIVAL OF OPEN-SOURCE PROJECTS: A POPULATION ECOLOGY PERSPECTIVE , 2003 .

[42]  William M. K. Trochim,et al.  Research methods knowledge base , 2001 .

[43]  Mohammad Alshayeb,et al.  An empirical study of system design instability metric and design evolution in an agile software process , 2005, J. Syst. Softw..

[44]  Phillip A. Laplante,et al.  Open Source Software: Is It Worth Converting? , 2007, IT Professional.

[45]  Tibor Gyimóthy,et al.  Extracting facts from open source software , 2004, 20th IEEE International Conference on Software Maintenance, 2004. Proceedings..

[46]  T. Rozman,et al.  Comparative assessment of open source software using easy accessible data , 2004, 26th International Conference on Information Technology Interfaces, 2004..

[47]  Georg von Krogh,et al.  The Promise of Research on Open Source Software , 2006, Manag. Sci..

[48]  Qiang Tu,et al.  Growth, evolution, and structural change in open source software , 2001, IWPSE '01.

[49]  David Baccarini,et al.  The Logical Framework Method for Defining Project Success , 1999 .

[50]  Likoebe M. Maruping,et al.  Impacts of License Choice and Organizational Sponsorship on User Interest and Development Activity in Open Source Software Projects , 2006, Inf. Syst. Res..

[51]  Hongfang Liu,et al.  Identifying and characterizing change-prone classes in two large-scale open-source products , 2007, J. Syst. Softw..

[52]  Stephen R. Schach,et al.  Measuring the maintainability of open-source software , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[53]  Kevin Crowston,et al.  Open source software projects as virtual organisations: competency rallying for software development , 2002, IEE Proc. Softw..

[54]  Chris DiBona,et al.  Open Sources: Voices from the Open Source Revolution , 1999 .

[55]  Stefan Koch,et al.  Effort, co‐operation and co‐ordination in an open source software project: GNOME , 2002, Inf. Syst. J..

[56]  Roger S Pressman.  Software Engineering: A Practitioner's Approach with Bonus Chapter on Agile Development , 2003 .

[57]  Jouni Markkula,et al.  Evaluating the Impact of Adaptive Maintenance Process on Open Source Software Quality , 2007, First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007).

[58]  David W. Hosmer,et al.  Applied Logistic Regression , 1991 .

[59]  Norman F. Schneidewind,et al.  Methodology For Validating Software Metrics , 1992, IEEE Trans. Software Eng..

[60]  Uzma Raja,et al.  Investigating Open Source Project Success: A Data Mining Approach to Model Formulation, Validation and Testing , 2006 .

[61]  Greg Madey,et al.  Understanding OSS as a Self-Organizing Process , 2002 .

[62]  Raed Shatnawi,et al.  Predicting Error Probability in the Eclipse Project , 2006, Software Engineering Research and Practice.

[63]  Elaine J. Weyuker,et al.  Evaluating Software Complexity Measures , 2010, IEEE Trans. Software Eng..

[64]  Eric A. von Hippel,et al.  How Open Source Software Works: 'Free' User-to-User Assistance? , 2000 .

[65]  J. Herbsleb,et al.  Two case studies of open source software development: Apache and Mozilla , 2002, TSEM.

[66]  J. Pierce.  An introduction to information theory: symbols, signals & noise , 1980 .