A Better Set of Object-Oriented Design Metrics for Within-Project Defect Prediction

Background: Using design metrics to predict fault-prone elements of a software design can help to focus attention on classes that need redesign and more extensive testing. However, some design metrics have been pointed out to be theoretically invalid, and the usefulness of some metrics is questioned. Aim: To identify a set of object-oriented metrics that are theoretically valid, and useful for identifying fault-prone classes in a design. Method: Drawing on four well-known sets of design metrics (CK, LK, MOOD and QMOOD), we propose a consolidated set of metrics that covers many aspects of object-oriented software design. We conduct two experiments, first using a single large system and then considering successive releases of that system, to compare the usefulness of the consolidated set with the other four sets for within-project prediction of fault-prone classes. Results: Both experiments suggest the consolidated set is effective at identifying fault-prone classes, outperforming the other metric sets (though at a cost of more false alarms). Conclusion: This paper adds to knowledge about the usefulness of existing sets of design metrics for within-project defect prediction, and identifies a consolidated set of metrics that is more effective than the existing sets at identifying fault-prone classes.

[1]  Mahmoud O. Elish,et al.  Empirical comparison of three metrics suites for fault prediction in packages of object-oriented systems: A case study of Eclipse , 2011, Adv. Eng. Softw..

[2]  Ruchika Malhotra,et al.  A systematic review of machine learning techniques for software fault prediction , 2015, Appl. Soft Comput..

[3]  Brian Henderson-Sellers,et al.  Object-Oriented Metrics , 1995, TOOLS.

[4]  Yuming Zhou,et al.  Empirical analysis of network measures for effort-aware fault-proneness prediction , 2016, Inf. Softw. Technol..

[5]  Alexander S. Yeh,et al.  More accurate tests for the statistical significance of result differences , 2000, COLING.

[6]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[7]  RadjenovićDanijel,et al.  Software fault prediction metrics , 2013 .

[8]  Ruchika Malhotra,et al.  Comparative analysis of statistical and machine learning methods for predicting faulty modules , 2014, Appl. Soft Comput..

[9]  Martin Hitz,et al.  Chidamber & Kemerer's Metrics Suite: a Measurement Theory Perspective , 1996 .

[10]  D. Hinkle,et al.  Applied statistics for the behavioral sciences , 1979 .

[11]  Tracy Hall,et al.  Measuring OO systems: a critical analysis of the MOOD metrics , 1999, Proceedings Technology of Object-Oriented Languages and Systems. TOOLS 29 (Cat. No.PR00275).

[12]  David W. Hosmer,et al.  Applied Logistic Regression , 1991 .

[13]  Ayse Basar Bener,et al.  On the relative value of cross-company and within-company data for defect prediction , 2009, Empirical Software Engineering.

[14]  Khaled El Emam,et al.  The Confounding Effect of Class Size on the Validity of Object-Oriented Metrics , 2001, IEEE Trans. Software Eng..

[15]  Mark Lorenz,et al.  Object-oriented software metrics - a practical guide , 1994 .

[16]  Guy Lapalme,et al.  A systematic analysis of performance measures for classification tasks , 2009, Inf. Process. Manag..

[17]  Victor R. Basili,et al.  A Validation of Object-Oriented Design Metrics as Quality Indicators , 1996, IEEE Trans. Software Eng..

[18]  Fernando Brito e Abreu,et al.  Object-Oriented Software Engineering: Measuring and Controlling the Development Process , 1994 .

[19]  Sandro Morasca,et al.  Property-Based Software Engineering Measurement , 1996, IEEE Trans. Software Eng..

[20]  Richard Torkar,et al.  Software fault prediction metrics: A systematic literature review , 2013, Inf. Softw. Technol..

[21]  Harald C. Gall,et al.  Cross-project defect prediction: a large scale experiment on data vs. domain vs. process , 2009, ESEC/SIGSOFT FSE.

[22]  Cagatay Catal,et al.  Software fault prediction: A literature review and current trends , 2011, Expert Syst. Appl..

[23]  James Allan,et al.  A comparison of statistical significance tests for information retrieval evaluation , 2007, CIKM '07.

[24]  Tracy Hall,et al.  A Systematic Literature Review on Fault Prediction Performance in Software Engineering , 2012, IEEE Transactions on Software Engineering.

[25]  Burak Turhan,et al.  A Systematic Literature Review and Meta-Analysis on Cross Project Defect Prediction , 2019, IEEE Transactions on Software Engineering.

[26]  Rachel Harrison,et al.  An overview of object-oriented design metrics , 1997, Proceedings Eighth IEEE International Workshop on Software Technology and Engineering Practice incorporating Computer Aided Software Engineering.

[27]  Tibor Gyimóthy,et al.  Empirical validation of object-oriented metrics on open source software for fault prediction , 2005, IEEE Transactions on Software Engineering.

[28]  Soumya Banerjee,et al.  Application of Artificial Immune Systems Paradigm for Developing Software Fault Prediction Models , 2010 .

[29]  Letha H. Etzkorn,et al.  Empirical Validation of Three Software Metrics Suites to Predict Fault-Proneness of Object-Oriented Classes Developed Using Highly Iterative or Agile Software Development Processes , 2007, IEEE Transactions on Software Engineering.

[30]  A. Zeller,et al.  Predicting Defects for Eclipse , 2007, Third International Workshop on Predictor Models in Software Engineering (PROMISE'07: ICSE Workshops 2007).

[31]  Sandro Morasca,et al.  Investigating the impact of fault data completeness over time on predicting class fault-proneness , 2017, Inf. Softw. Technol..

[32]  Javam C. Machado,et al.  The prediction of faulty classes using object-oriented design metrics , 2001, J. Syst. Softw..

[33]  Yann-Gaël Guéhéneuc,et al.  Design evolution metrics for defect prediction in object oriented systems , 2010, Empirical Software Engineering.

[34]  Mei-Hwa Chen,et al.  An empirical study on object-oriented metrics , 1999, Proceedings Sixth International Software Metrics Symposium (Cat. No.PR00403).

[35]  Yuming Zhou,et al.  Are Slice-Based Cohesion Metrics Actually Useful in Effort-Aware Post-Release Fault-Proneness Prediction? An Empirical Study , 2015, IEEE Transactions on Software Engineering.

[36]  Banu Diri,et al.  A systematic review of software fault prediction studies , 2009, Expert Syst. Appl..

[37]  Richard H. Carver,et al.  An Evaluation of the MOOD Set of Object-Oriented Software Metrics , 1998, IEEE Trans. Software Eng..

[38]  Ayça Tarhan,et al.  Early software defect prediction: A systematic map and review , 2018, J. Syst. Softw..

[39]  Carl G. Davis,et al.  A Hierarchical Model for Object-Oriented Design Quality Assessment , 2002, IEEE Trans. Software Eng..

[40]  Wei Li,et al.  Another metric suite for object-oriented programming , 1998, J. Syst. Softw..