The influence of organizational structure on software quality

Often software systems are developed by organizations consisting of many teams of individuals working together. Brooks states in the Mythical Man Month book that product quality is strongly affected by organization structure. Unfortunately there has been little empirical evidence to date to substantiate this assertion. In this paper we present a metric scheme to quantify organizational complexity, in relation to the product development process to identify if the metrics impact failure-proneness. In our case study, the organizational metrics when applied to data from Windows Vista were statistically significant predictors of failure-proneness. The precision and recall measures for identifying failure-prone binaries, using the organizational metrics, was significantly higher than using traditional metrics like churn, complexity, coverage, dependencies, and pre-release bug measures that have been used to date to predict failure-proneness. Our results provide empirical evidence that the organizational metrics are related to, and are effective predictors of failure-proneness.

[1]  Padmanabhan Santhanam,et al.  Exploring defect data from development and customer usage on software modules over multiple releases , 1998, Proceedings Ninth International Symposium on Software Reliability Engineering (Cat. No.98TB100257).

[2]  Andreas Zeller,et al.  Mining metrics to predict component failures , 2006, ICSE.

[3]  Audris Mockus,et al.  Drivers for Customer Perceived Software Quality , 2005, ICSE 2005.

[4]  J. Edward Jackson,et al.  A User's Guide to Principal Components. , 1991 .

[5]  Elliot Soloway,et al.  Where the bugs are , 1985, CHI '85.

[6]  J. Herbsleb,et al.  Two case studies of open source software development: Apache and Mozilla , 2002, TSEM.

[7]  D. Kleinbaum,et al.  Applied Regression Analysis and Other Multivariate Methods , 1978 .

[8]  Leon J. Osterweil,et al.  Software processes are software too , 1987, ISPW.

[9]  Dewayne E. Perry,et al.  People, organizations, and process improvement , 1994, IEEE Software.

[10]  Shari Lawrence Pfleeger,et al.  Software Metrics , 1991 .

[11]  Lori A. Clarke,et al.  A Formal Model of Program Dependences and Its Implications for Software Testing, Debugging, and Maintenance , 1990, IEEE Trans. Software Eng..

[12]  D HerbslebJames,et al.  Two case studies of open source software development , 2002 .

[13]  Ping Zhang,et al.  Predictors of customer perceived software quality , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[14]  Taghi M. Khoshgoftaar,et al.  Improving code churn predictions during the system test and maintenance phases , 1994, Proceedings 1994 International Conference on Software Maintenance.

[15]  James D. Herbsleb,et al.  Architectures, coordination, and distance: Conway’s law and beyond , 1999 .

[16]  N. Nagappan,et al.  Use of relative code churn measures to predict system defect density , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[17]  James D. Herbsleb,et al.  Splitting the organization and integrating the code: Conway's law revisited , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[18]  Andreas Zeller,et al.  Mining Version Histories to Guide Software Changes , 2004 .

[19]  D. L. Parnas,et al.  On the criteria to be used in decomposing systems into modules , 1972, Software Pioneers.

[20]  Andreas Zeller,et al.  Predicting component failures at design time , 2006, ISESE '06.

[21]  Xiuzhen Zhang,et al.  Comments on "Data Mining Static Code Attributes to Learn Defect Predictors" , 2007, IEEE Trans. Software Eng..

[22]  Jr. Frederick P. Brooks,et al.  The mythical man-month (anniversary ed.) , 1995 .

[23]  Phyllis G. Frankl,et al.  An Experimental Comparison of the Effectiveness of Branch Testing and Data Flow Testing , 1993, IEEE Trans. Software Eng..

[24]  Thomas J. Ostrand,et al.  Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria , 1994, Proceedings of 16th International Conference on Software Engineering.

[25]  Tim Menzies,et al.  Data Mining Static Code Attributes to Learn Defect Predictors , 2007 .

[26]  Harvey P. Siy,et al.  Predicting Fault Incidence Using Software Change History , 2000, IEEE Trans. Software Eng..

[27]  Taghi M. Khoshgoftaar,et al.  Detection of software modules with high debug code churn in a very large legacy system , 1996, Proceedings of ISSRE '96: 7th International Symposium on Software Reliability Engineering.

[28]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[29]  Forrest Shull,et al.  Building Knowledge through Families of Experiments , 1999, IEEE Trans. Software Eng..

[30]  Audris Mockus,et al.  Formulation and preliminary test of an empirical theory of coordination in software engineering , 2003, ESEC/FSE-11.

[31]  Victor R. Basili,et al.  A Validation of Object-Oriented Design Metrics as Quality Indicators , 1996, IEEE Trans. Software Eng..

[32]  Elaine J. Weyuker,et al.  Predicting the location and number of faults in large software systems , 2005, IEEE Transactions on Software Engineering.

[33]  Fred P. Brooks,et al.  The Mythical Man-Month , 1975, Reliable Software.

[34]  M. E. Conway.  HOW DO COMMITTEES INVENT , 1967 .

[35]  Nachiappan Nagappan,et al.  Using Software Dependencies and Churn Metrics to Predict Field Failures: An Empirical Case Study , 2007, First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007).

[36]  Lionel C. Briand,et al.  Investigating quality factors in object-oriented designs: an industrial case study , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[37]  J. E. Jackson.  A User's Guide to Principal Components , 1991 .

[38]  G. Denaro,et al.  An empirical evaluation of fault-proneness models , 2002, Proceedings of the 24th International Conference on Software Engineering. ICSE 2002.

[39]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[40]  Brendan Murphy,et al.  Using Historical In-Process and Product Metrics for Early Estimation of Software Failures , 2006, 2006 17th International Symposium on Software Reliability Engineering.