Analyzing Software Quality Evolution using Metrics: An Empirical Study on Open Source Software

The study presented in this paper aims at analyzing empirically the quality of evolving software systems using metrics. We used a synthetic metric (Quality Assurance Indicator - Qi), which captures in an integrated way different object-oriented software attributes. We wanted to investigate if the Qi metric can be used to observe how quality evolves along the evolution of software systems. We consider software quality from an internal (structural) perspective. We used various object-oriented design metrics for measuring the structural quality of a release. We performed an empirical analysis using historical data collected from successive released versions of three open source (Java) software systems. The collected data cover, for each system, a period of several years (4 years for two systems and 7 years for the third one). We focused on three issues: (1) the evolution of the Qi metric along the evolution of the subject systems, (2) the class growth of the subject systems, and (3) the quality of added classes versus the quality of removed ones. Empirical results provide evidence that the Qi metric reflects properly the quality evolution of the studied software systems.

[1]  Yuming Zhou,et al.  Empirical Analysis of Object-Oriented Design Metrics for Predicting High and Low Severity Faults , 2006, IEEE Transactions on Software Engineering.

[2]  Ramanath Subramanyam,et al.  Empirical Analysis of CK Metrics for Object-Oriented Design Complexity: Implications for Software Defects , 2003, IEEE Trans. Software Eng..

[3]  Sandro Pinna,et al.  Empirical study of software quality evolution in open source projects using agile practices , 2009, ArXiv.

[4]  N. Drouin,et al.  On understanding software quality evolution from a defect perspective: A case study on an open source software system , 2012, 2012 International Conference on Computer Systems and Industrial Informatics.

[5]  Meir M. Lehman,et al.  Program evolution: processes of software change , 1985 .

[6]  Andreas Zeller,et al.  Mining metrics to predict component failures , 2006, ICSE.

[7]  Arvinder Kaur,et al.  Empirical validation of object-oriented metrics for predicting fault proneness models , 2010, Software Quality Journal.

[8]  Xiao Ming Yuan,et al.  Distribution and characteristics of gravelly soil liquefaction in the Wenchuan Ms 8.0 earthquake , 2010 .

[9]  Jan Bosch,et al.  Design Erosion: Problems & Causes , 2001 .

[10]  Norman E. Fenton,et al.  Software Metrics: A Rigorous Approach , 1991 .

[11]  David P. Darcy,et al.  Managerial Use of Metrics for Object-Oriented Software: An Exploratory Analysis , 1998, IEEE Trans. Software Eng..

[12]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[13]  Juan Fernández-Ramil,et al.  The evolution of Eclipse , 2008, 2008 IEEE International Conference on Software Maintenance.

[14]  Sinan Eski,et al.  An Empirical Study on Object-Oriented Metrics and Software Evolution in Order to Reduce Testing Costs by Predicting Change-Prone Classes , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[15]  David Lorge Parnas,et al.  Software aging , 1994, Proceedings of 16th International Conference on Software Engineering.

[16]  Michele Marchesi,et al.  Studying the Evolution of Quality Metrics in an Agile/Distributed Project , 2006, XP.

[17]  Mourad Badri,et al.  Metrics and Software Quality Evolution: A Case Study on Open Source Software , 2013 .

[18]  Iulian Neamtiu,et al.  Towards a better understanding of software evolution: An empirical study on open source software , 2009, 2009 IEEE International Conference on Software Maintenance.

[19]  Jens H. Weber,et al.  Predicting maintainability with object-oriented metrics -an empirical comparison , 2003, 10th Working Conference on Reverse Engineering, 2003. WCRE 2003. Proceedings..

[20]  Mourad Badri,et al.  Evaluating the Effect of Control Flow on the Unit Testing Effort of Classes: An Empirical Analysis , 2012, Adv. Softw. Eng..

[21]  Young Lee,et al.  Metrics and Evolution in Open Source Software , 2007 .

[22]  Dewayne E. Perry,et al.  Metrics and laws of software evolution-the nineties view , 1997, Proceedings Fourth International Software Metrics Symposium.

[23]  Mourad Badri,et al.  Empirical Analysis for Investigating the Effect of Control Flow Dependencies on Testability of Classes , 2011, SEKE.

[24]  Tom Mens,et al.  Future trends in software evolution metrics , 2001, IWPSE '01.

[25]  Meir M. Lehman,et al.  On understanding laws, evolution, and conservation in the large-program life cycle , 1984, J. Syst. Softw..

[26]  Arvinder Kaur,et al.  Empirical analysis for investigating the effect of object-oriented metrics on fault proneness: a replicated case study , 2009 .

[27]  Hongyu Zhang,et al.  Monitoring Software Quality Evolution for Defects , 2010, IEEE Software.

[28]  Deepak Goyal,et al.  A hierarchical model for object-oriented design quality assessment , 2015 .

[29]  Srini Ramaswamy,et al.  Using bug reports as a software quality measure , 2011, ICIQ.

[30]  Meir M. Lehman,et al.  Laws of Software Evolution Revisited , 1996, EWSPT.

[31]  Brian Henderson-Sellers,et al.  Object-Oriented Metrics , 1995, TOOLS.

[32]  Marco Scotto,et al.  Visual identification of software evolution patterns , 2007, IWPSE '07.

[33]  Lionel C. Briand,et al.  Exploring the relationships between design measures and software quality in object-oriented systems , 2000, J. Syst. Softw..

[34]  Mourad Badri,et al.  Empirical Analysis of Object-Oriented Design Metrics: Towards a New Metric Using Control Flow Paths and Probabilities , 2009, J. Object Technol..

[35]  Victor R. Basili,et al.  A Validation of Object-Oriented Design Metrics as Quality Indicators , 1996, IEEE Trans. Software Eng..