Towards Meaningful Software Metrics Aggregation

Aggregation of software metrics is a challenging task, it is even more complex when it comes to considering weights to indicate the relative importance of software metrics. These weights are mostly determined manually, it results in subjective quality models, which are hard to interpret. To address this challenge, we propose an automated aggregation approach based on the joint distribution of software metrics. To evaluate the effectiveness of our approach, we conduct an empirical study on maintainability assessment for around 5 000 classes from open source software systems written in Java and compare our approach with a classical weighted linear combination approach in the context of maintainability scoring and anomaly detection. The results show that approaches assign similar scores, while our approach is more interpretable, sensitive, and actionable.

[1]  Tibor Gyimóthy,et al.  A probabilistic software quality model , 2011, 2011 27th IEEE International Conference on Software Maintenance (ICSM).

[2]  Joost Visser,et al.  Benchmark-Based Aggregation of Metrics to Ratings , 2011, 2011 Joint Conference of the 21st International Workshop on Software Measurement and the 6th International Conference on Software Process and Product Measurement.

[3]  Audris Mockus,et al.  Thresholds for Size and Complexity Metrics: A Case Study from the Perspective of Defect Density , 2016, 2016 IEEE International Conference on Software Quality, Reliability and Security (QRS).

[4]  Marco Tulio Valente,et al.  RTTool: A Tool for Extracting Relative Thresholds for Source Code Metrics , 2014, 2014 IEEE International Conference on Software Maintenance and Evolution.

[5]  D. Altman,et al.  Measuring agreement in method comparison studies , 1999, Statistical methods in medical research.

[6]  Nicolas Anquetil,et al.  Software quality metrics aggregation in industry , 2013, J. Softw. Evol. Process..

[7]  Joost Visser,et al.  A Practical Model for Measuring Maintainability , 2007, 6th International Conference on the Quality of Information and Communications Technology (QUATIC 2007).

[8]  Charles A. Sutton,et al.  Mining source code repositories at massive scale using language modeling , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[9]  Thierry Coq,et al.  The SQALE Analysis Model: An Analysis Model Compliant with the Representation Condition for Assessing the Quality of Software Source Code , 2010, 2010 Second International Conference on Advances in System Testing and Validation Lifecycle.

[10]  Philip H. Ramsey Nonparametric Statistical Methods , 1974, Technometrics.

[11]  Bill Ravens,et al.  An Introduction to Copulas , 2000, Technometrics.

[12]  Alexander K. Petrenko,et al.  Certification of Technical Quality of Software Products , 2008 .

[13]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[14]  Oscar Nierstrasz,et al.  Comparative analysis of evolving software systems using the Gini coefficient , 2009, 2009 IEEE International Conference on Software Maintenance.

[15]  Joost Visser,et al.  Standardized code quality benchmarking for improving software maintainability , 2011, Software Quality Journal.

[16]  Carl G. Davis,et al.  A Hierarchical Model for Object-Oriented Design Quality Assessment , 2002, IEEE Trans. Software Eng..

[17]  Wm. R. Wright General Intelligence, Objectively Determined and Measured. , 1905 .

[18]  Alexander Serebrenik,et al.  Theil index for aggregation of software metrics values , 2010, 2010 IEEE International Conference on Software Maintenance.

[19]  Alexander Serebrenik,et al.  By no means: a study on aggregating software metrics , 2011, WETSoM '11.

[20]  Stéphane Ducasse,et al.  The squale model — A practice-based industrial quality model , 2009, 2009 IEEE International Conference on Software Maintenance.

[21]  R. Forthofer,et al.  Rank Correlation Methods , 1981 .

[22]  Reinhold Plösch,et al.  Operationalised product quality models and assessment: The Quamoco approach , 2014, Inf. Softw. Technol..

[23]  Morgan Ericsson,et al.  Quality Models Inside Out: Interactive Visualization of Software Metrics by Means of Joint Probabilities , 2018, 2018 IEEE Working Conference on Software Visualization (VISSOFT).