Measuring the Effectiveness of the Defect-Fixing Process in Open Source Software Projects

The defect-fixing process is a key process in which an open source software (OSS) project team responds to customer needs in terms of detecting and resolving software defects, hence the dimension of defect-fixing effectiveness corresponds nicely to adopters' concerns regarding OSS products. Although researchers have been studying the defect fixing process in OSS projects for almost a decade, the literature still lacks rigorous ways to measure the effectiveness of this process. Thus, this paper aims to create a valid and reliable instrument to measure the defect-fixing effectiveness construct in an open source environment through the scale development methodology proposed by Churchill [4]. This paper examines the validity and reliability of an initial list of indicators through two rounds of data collection and analysis. Finally four indicators are suggested to measure defect-fixing effectiveness. The implication for practitioners is explained through a hypothetical example followed by implications for the research community.

[1]  Tzvi Raz,et al.  Factors affecting design inspection effectiveness in software development , 1997, Inf. Softw. Technol..

[2]  Kevin Crowston,et al.  Information systems success in free and open source software development: theory and measures , 2006, Softw. Process. Improv. Pract..

[3]  Lee Sproull,et al.  Design for quality: the case of open source software development , 2008 .

[4]  Ju Long,et al.  Understanding the Role of Core Developers in Open Source Software Development , 2006 .

[5]  Gilbert A. Churchill.  A Paradigm for Developing Better Measures of Marketing Constructs , 1979 .

[6]  James A. Senn,et al.  Challenges and strategies for research in systems development , 1992 .

[7]  Audris Mockus,et al.  Interval Quality: Relating Customer-Perceived Quality to Process Quality , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[8]  C. Wiseman.  Strategic Information Systems: Trends and Challenges over the Next Decade. , 1988 .

[9]  Rolph E. Anderson,et al.  Multivariate data analysis (4th ed.): with readings , 1995 .

[10]  Trevor John Cockram,et al.  The use of Bayesian networks to determine software inspection process efficiency , 2001 .

[11]  Maurizio Morisio,et al.  Characteristics of open source projects , 2003, Seventh European Conference onSoftware Maintenance and Reengineering, 2003. Proceedings..

[12]  Nader B. Ebrahimi,et al.  On the Statistical Analysis of the Number of Errors Remaining in a Software Design Document after Inspection , 1997, IEEE Trans. Software Eng..

[13]  Duane T. Wegener,et al.  Evaluating the use of exploratory factor analysis in psychological research. , 1999 .

[14]  Horst Remus,et al.  Integrated software validation in the view of inspections/reviews , 1984 .

[15]  Edward Yourdon,et al.  Death March , 1997 .

[16]  Bernard Golden,et al.  Succeeding with Open Source , 2004 .

[17]  Oliver Laitenberger,et al.  An encompassing life cycle centric survey of software inspection , 2000, J. Syst. Softw..

[18]  B. Walters.  Understanding Open Source Software , 2006 .

[19]  Prashant Palvia,et al.  Antecedent to the success of open source software , 2007 .

[20]  Kevin Crowston,et al.  Bug Fixing Practices within Free/Libre Open Source Software Development Teams , 2008, J. Database Manag..

[21]  Katherine J. Stewart,et al.  The moderating role of development stage in free/open source software project performance , 2006, Softw. Process. Improv. Pract..

[22]  Michael E. Fagan.  Design and Code Inspections to Reduce Errors in Program Development , 1976, IBM Syst. J..

[23]  Rolph E. Anderson,et al.  Multivariate Data Analysis with Readings , 1979 .

[24]  Moez Limayem,et al.  How Habit Limits the Predictive Power of Intention: The Case of Information Systems Continuance , 2007, MIS Q..

[25]  Katherine J. Stewart,et al.  The Impact of Ideology on Effectiveness in Open Source Software Development Teams , 2006, MIS Q..

[26]  Tzvi Raz,et al.  Inspection effectiveness in software development: a neural network approach , 1994, CASCON.

[27]  Audris Mockus,et al.  An Empirical Study of Speed and Communication in Globally Distributed Software Development , 2003, IEEE Trans. Software Eng..

[28]  Yuan Long,et al.  Social structure, knowledge sharing, and project performance in open source software development , 2006 .

[29]  Vahid Garousi.  Evidence-Based Insights about Issue Management Processes: An Exploratory Study , 2009, ICSP.

[30]  Claes Wohlin,et al.  Defect content estimations from review data , 1998, Proceedings of the 20th International Conference on Software Engineering.

[31]  Kevin Crowston,et al.  Defining Open Source Software Project Success , 2003, ICIS.

[32]  Claes Wohlin,et al.  The impact of time controlled reading on software inspection effectiveness and efficiency: a controlled experiment , 2008, ESEM '08.

[33]  Bernard Golden.  Succeeding with Open Source (Addison-Wesley Information Technology Series) , 2004 .

[34]  Ju Long.  Understanding the creation and adoption of information technology innovations: the case of open source software development and the diffusion of mobile commerce , 2004 .

[35]  Brian Fitzgerald,et al.  Understanding open source software development , 2002 .

[36]  Kevin Crowston,et al.  Open source software projects as virtual organisations: competency rallying for software development , 2002, IEE Proc. Softw..

[37]  Scott N. Woodfield,et al.  Evaluating the effectiveness of reliability-assurance techniques , 1989, J. Syst. Softw..

[38]  J. Hair.  Multivariate data analysis , 1972 .