An Empirical Study on the Relationship among Software Design Quality , Development Effort , and Governance in Open Source Projects

The relationship among software design quality, development effort, and governance practices is a traditional research problem. However, the extent to which consolidated results on this relationship remain valid for open source (OS) projects is an open research problem. An emerging body of literature contrasts the view of open source as an alternative to proprietary software and explains that there exists a continuum between closed and open source projects. This paper hypothesizes that as projects approach the OS end of the continuum, governance becomes less formal. In turn a less formal governance is hypothesized to require a higher-quality code as a means to facilitate coordination among developers by making the structure of code explicit and facilitate quality by removing the pressure of deadlines from contributors. However, a less formal governance is also hypothesized to increase development effort due to a more cumbersome coordination overhead. The verification of research hypotheses is based on empirical data from a sample of 75 major OS projects. Empirical evidence supports our hypotheses and suggests that software quality, mainly measured as coupling and inheritance, does not increase development effort, but represents an important managerial variable to implement the more open governance approach that characterizes OS projects which, in turn, increases development effort.

[1]  K. Beck,et al.  Extreme Programming Explained , 2002 .

[2]  Hyunsoo Kim,et al.  The software maintenance project effort estimation model based on function points , 2003, J. Softw. Maintenance Res. Pract..

[3]  Sandra Slaughter,et al.  Communication Networks in an Open Source Software Project , 2006, OSS.

[4]  Juan Carlos Fernandez Ramil.  Continual resource estimation for evolving software , 2003, International Conference on Software Maintenance, 2003. ICSM 2003. Proceedings..

[5]  J. Herbsleb,et al.  A systematic survey of CMM experience and results , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[6]  Sandeep Krishnamurthy.  A Managerial Overview of Open Source Software , 2003 .

[7]  Edward V. Thomas,et al.  Development of Robust Multivariate Calibration Models , 2000, Technometrics.

[8]  Stuart H. Zweben,et al.  Measuring the quality of structured designs , 1981, J. Syst. Softw..

[9]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[10]  Ramanath Subramanyam,et al.  Empirical Analysis of CK Metrics for Object-Oriented Design Complexity: Implications for Software Defects , 2003, IEEE Trans. Software Eng..

[11]  Jeffrey A. Sonnenfeld,et al.  Good governance and the misleading myths of bad metrics , 2004 .

[12]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[13]  Brian Fitzgerald,et al.  A Critical Look at Open Source , 2004, Computer.

[14]  James E. Tomayko,et al.  The structural complexity of software an experimental test , 2005, IEEE Transactions on Software Engineering.

[15]  L. Kirsch.  The Management of Complex Tasks in Organizations: Controlling the Systems Development Process , 1996 .

[16]  Mattia Monga,et al.  From Bazaar to Kibbutz: How Freedom Deals with Coherence in the Debian Project , 2004, ICSE 2004.

[17]  M A Branch,et al.  Software maintenance management , 1986 .

[18]  Eric S. Raymond,et al.  The Art of Unix Programming , 2003 .

[19]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[20]  Ilkka Tervonen,et al.  Towards deeper co-understanding of software quality , 1998, Inf. Softw. Technol..

[21]  Stefan Koch,et al.  Agile Principles and Open Source Software Development: A Theoretical and Empirical Discussion , 2004, XP.

[22]  Mary Beth Chrissis,et al.  CMMI: Guidelines for Process Integration and Product Improvement , 2003 .

[23]  Chris F. Kemerer,et al.  An empirical validation of software cost estimation models , 1987, CACM.

[24]  Maurice H. Halstead,et al.  Elements of software science , 1977 .

[25]  Ellis Horowitz,et al.  Software Cost Estimation with COCOMO II , 2000 .

[26]  Tom Mens,et al.  Future trends in software evolution metrics , 2001, IWPSE '01.

[27]  Richard H. Carver,et al.  An Evaluation of the MOOD Set of Object-Oriented Software Metrics , 1998, IEEE Trans. Software Eng..

[28]  Kevin Crowston,et al.  The social structure of free and open source software development , 2005, First Monday.

[29]  Sallie M. Henry,et al.  Maintenance metrics for the object oriented paradigm , 1993, [1993] Proceedings First International Software Metrics Symposium.

[30]  Brian Fitzgerald,et al.  The Transformation of Open Source Software , 2006, MIS Q..

[31]  E. Carmines,et al.  Analyzing models with unobserved variables: analysis of covariance structures , 1981 .

[32]  Manish Agrawal,et al.  Software Effort, Quality, and Cycle Time: A Study of CMM Level 5 Projects , 2007, IEEE Transactions on Software Engineering.

[33]  Richard P. Gabriel,et al.  Innovation happens elsewhere - open source as business strategy , 2005 .

[34]  Siu Leung Chung,et al.  An Economic Model to Estimate Software Rewriting and Replacement Times , 1996, IEEE Trans. Software Eng..

[35]  A.I. Wasserman,et al.  Evaluating Software Engineering Processes in Commercial and Community Open Source Projects , 2007, First International Workshop on Emerging Trends in FLOSS Research and Development (FLOSS'07: ICSE Workshops 2007).

[36]  Romain Robbes.  Mining a Change-Based Software Repository , 2007, Fourth International Workshop on Mining Software Repositories (MSR'07:ICSE Workshops 2007).

[37]  Roger B. Stake,et al.  A Framework for Evaluating Commonality , 2001 .

[38]  Alan MacCormack,et al.  Exploring the Structure of Complex Software Designs: An Empirical Study of Open Source and Proprietary Code , 2006, Manag. Sci..

[39]  David F. Larcker,et al.  Structural Equation Models with Unobservable Variables and Measurement Error: Algebra and Statistics: , 1981 .

[40]  Abraham Bernstein,et al.  Improving defect prediction using temporal features and non linear models , 2007, IWPSE '07.

[41]  Rob Procter,et al.  Software Development Practices , 1995 .

[42]  Roy T. Fielding,et al.  Shared leadership in the Apache project , 1999, CACM.

[43]  Tom Mens,et al.  A survey of software refactoring , 2004, IEEE Transactions on Software Engineering.

[44]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[45]  D. Kaplan.  Structural Equation Modeling: Foundations and Extensions , 2000 .

[46]  Lionel C. Briand,et al.  A Unified Framework for Coupling Measurement in Object-Oriented Systems , 1999, IEEE Trans. Software Eng..

[47]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[48]  Jaak Jurison,et al.  Software Project Management: The Manager's View , 1999, Commun. Assoc. Inf. Syst..

[49]  Ned Chapin,et al.  Types of software evolution and software maintenance , 2001, J. Softw. Maintenance Res. Pract..

[50]  Rex B. Kline,et al.  Principles and Practice of Structural Equation Modeling , 1998 .

[51]  Michiel van Genuchten,et al.  Why is Software Late? An Empirical Study of Reasons For Delay in Software Development , 1991, IEEE Trans. Software Eng..

[52]  Sallie M. Henry,et al.  Software Structure Metrics Based on Information Flow , 1981, IEEE Transactions on Software Engineering.

[53]  Lois W. Sayrs.  Interviews : an introduction to qualitative research interviewing , 1996 .

[54]  Michele Lanza,et al.  Object-Oriented Metrics in Practice - Using Software Metrics to Characterize, Evaluate, and Improve the Design of Object-Oriented Systems , 2006 .

[55]  Martin Hitz,et al.  Chidamber & Kemerer's Metrics Suite: a Measurement Theory Perspective , 1996 .

[56]  C. Fornell,et al.  Evaluating Structural Equation Models with Unobservable Variables and Measurement Error , 1981 .

[57]  Audris Mockus,et al.  Identifying Productivity Drivers by Modeling Work Units Using Partial Data , 2001, Technometrics.

[58]  Alan Boulanger.  Open-source versus proprietary software: Is one more reliable and secure than the other? , 2005, IBM Syst. J..

[59]  Mik Kersten,et al.  Using task context to improve programmer productivity , 2006, SIGSOFT '06/FSE-14.

[60]  Chiara Francalanci,et al.  The Economics of Open Source Software: An Empirical Analysis of Maintenance Costs , 2007, 2007 IEEE International Conference on Software Maintenance.

[61]  Kevin Crowston,et al.  The Perils and Pitfalls of Mining SourceForge , 2004, MSR.

[62]  Mayuram S. Krishnan,et al.  Evaluating the cost of software quality , 1998, CACM.

[63]  Martin Michlmayr,et al.  USENIX Association Proceedings of the FREENIX Track : 2004 , 2004 .

[64]  Kaoru Ishikawa.  Introduction to Quality Control , 1990 .

[65]  Gordon B. Davis,et al.  Software Development Practices, Software Complexity, and Software Maintenance Performance: a Field Study , 1998 .

[66]  Mutsumi Komuro,et al.  Experiences of applying SPC techniques to software development processes , 2006, ICSE.

[67]  Rudiger Lincke,et al.  Compendium of Software Quality Standards and Metrics - Version 1.0 , 2007 .

[68]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[69]  Tibor Gyimóthy,et al.  Empirical validation of object-oriented metrics on open source software for fault prediction , 2005, IEEE Transactions on Software Engineering.

[70]  Shari Lawrence Pfleeger,et al.  Software Metrics : A Rigorous and Practical Approach , 1998 .

[71]  David P. Darcy,et al.  Managerial Use of Metrics for Object-Oriented Software: An Exploratory Analysis , 1998, IEEE Trans. Software Eng..

[72]  Andreas Zeller,et al.  Mining metrics to predict component failures , 2006, ICSE.

[73]  R. Bagozzi,et al.  On the evaluation of structural equation models , 1988 .

[74]  Abraham Bernstein,et al.  Predicting defect densities in source code files with decision tree learners , 2006, MSR '06.

[75]  Shari Lawrence Pfleeger,et al.  Software metrics (2nd ed.): a rigorous and practical approach , 1997 .

[76]  Victor R. Basili,et al.  Understanding and predicting the process of software maintenance releases , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[77]  Dirk Riehle,et al.  The Economic Motivation of Open Source Software: Stakeholder Perspectives , 2007, Computer.

[78]  Yong Tan,et al.  Comparing uniform and flexible policies for software maintenance and replacement , 2005, IEEE Transactions on Software Engineering.

[79]  Mr. Larry,et al.  Applying and Interpreting Object Oriented Metrics , 1998 .

[80]  Mayuram S. Krishnan,et al.  Effects of Process Maturity on Quality, Cycle Time, and Effort in Software Product Development , 2000 .

[81]  Rajiv D. Banker,et al.  A study of the effects of software development practices on software maintenance effort , 1996, 1996 Proceedings of International Conference on Software Maintenance.

[82]  Victor R. Basili,et al.  A Controlled Experiment Quantitatively Comparing Software Development Approaches , 1981, IEEE Transactions on Software Engineering.

[83]  Alessandro Bianchi,et al.  Evaluating software degradation through entropy , 2001, Proceedings Seventh International Software Metrics Symposium.

[84]  Victor R. Basili,et al.  A Validation of Object-Oriented Design Metrics as Quality Indicators , 1996, IEEE Trans. Software Eng..

[85]  L. Tucker,et al.  A reliability coefficient for maximum likelihood factor analysis , 1973 .

[86]  Patrick S. Renz.  Project Governance: Implementing Corporate Governance and Business Ethics in Nonprofit Organizations , 2007 .

[87]  Anthony I. Wasserman,et al.  A Framework for Evaluating Managerial Styles in Open Source Projects , 2008, OSS.

[88]  G. Goth.  Open source business models: ready for prime time , 2005, IEEE Software.

[89]  Anol Bhattacherjee,et al.  Understanding Information Systems Continuance: An Expectation-Confirmation Model , 2001, MIS Q..

[90]  J.-Y. Chen,et al.  A new metric for object-oriented design , 1993, Inf. Softw. Technol..

[91]  Robert C. Sharble,et al.  The object-oriented brewery: a comparison of two object-oriented development methods , 1993, SOEN.

[92]  Charles R. Symons,et al.  Function Point Analysis: Difficulties and Improvements , 1988, IEEE Trans. Software Eng..

[93]  Mik Kersten,et al.  Mylar: a degree-of-interest model for IDEs , 2005, AOSD '05.

[94]  Sandro Morasca,et al.  On the application of measurement theory in software engineering , 2004, Empirical Software Engineering.

[95]  Yuming Zhou,et al.  Empirical Analysis of Object-Oriented Design Metrics for Predicting High and Low Severity Faults , 2006, IEEE Transactions on Software Engineering.

[96]  Stephen R. Schach,et al.  Validation of the coupling dependency metric as a predictor of run-time failures and maintenance measures , 1998, Proceedings of the 20th International Conference on Software Engineering.