Software Product Quality Metrics: A Systematic Mapping Study

In the current competitive world, producing quality products has become a prominent factor to succeed in business. In this respect, defining and following the software product quality metrics (SPQM) to detect the current quality situation and continuous improvement of systems have gained tremendous importance. Therefore, it is necessary to review the present studies in this area to allow for the analysis of the situation at hand, as well as to enable us to make predictions regarding the future research areas. The present research aims to analyze the active research areas and trends on this topic appearing in the literature during the last decade. A Systematic Mapping (SM) study was carried out on 70 articles and conference papers published between 2009 and 2019 on SPQM as indicated in their titles and abstract. The result is presented through graphics, explanations, and the mind mapping method. The outputs include the trend map between the years 2009 and 2019, knowledge about this area and measurement tools, issues determined to be open to development in this area, and conformity between conference papers, articles and internationally valid quality models. This study may serve as a foundation for future studies that aim to contribute to the development in this crucial field. Future SM studies might focus on this subject for measuring the quality of network performance and new technologies such as Artificial Intelligence (AI), Internet of things (IoT), Cloud of Things (CoT), Machine Learning, and Robotics.

[1]  Audris Mockus,et al.  Questioning software maintenance metrics: A comparative case study , 2012, Proceedings of the 2012 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement.

[2]  Arashdeep Kaur,et al.  Early Software Fault Prediction Using Real Time Defect Data , 2009, 2009 Second International Conference on Machine Vision.

[3]  Claes Wohlin,et al.  Guidelines for snowballing in systematic literature studies and a replication in software engineering , 2014, EASE '14.

[4]  Ling Xu,et al.  A Systematic Mapping Study of Quality Assessment Models for Software Products , 2017, 2017 International Conference on Software Analysis, Testing and Evolution (SATE).

[5]  Reinhold Plösch,et al.  MUSE: A Framework for Measuring Object-Oriented Design Quality , 2016, J. Object Technol..

[6]  Mohammad Zulkernine,et al.  Using complexity, coupling, and cohesion metrics as early indicators of vulnerabilities , 2011, J. Syst. Archit..

[7]  Baris Özkan,et al.  ASSIST: An Integrated Measurement Tool , 2013, 2013 Joint Conference of the 23rd International Workshop on Software Measurement and the 8th International Conference on Software Process and Product Measurement.

[8]  Reinhard Schwarz,et al.  A Critical Survey of Security Indicator Approaches , 2012, 2012 Seventh International Conference on Availability, Reliability and Security.

[9]  Rouven Krebs,et al.  Automated inference of goal-oriented performance prediction functions , 2012, 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering.

[10]  Partha Sarathi Chakraborty,et al.  Software Component Quality Characteristics Model for Component Based Software Engineering , 2015, 2015 IEEE International Conference on Computational Intelligence & Communication Technology.

[11]  Jitender Kumar Chhabra,et al.  Package level cohesion measurement in object-oriented software , 2011, Journal of the Brazilian Computer Society.

[12]  Carlos Albuquerque,et al.  Metrics focused on usability ISO 9126 based , 2016, 2016 11th Iberian Conference on Information Systems and Technologies (CISTI).

[13]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[14]  Danilo Santos,et al.  External Quality Metrics for Object-Oriented Software: A Systematic Literature Review , 2017, CLEI Electron. J..

[15]  Hao Wang,et al.  Security metrics for software systems , 2009, ACM-SE 47.

[16]  Umesh L. Kulkarni,et al.  Validation of CK Metrics for Object Oriented Design Measurement , 2010, 2010 3rd International Conference on Emerging Trends in Engineering and Technology.

[17]  Pearl Brereton,et al.  Using Mapping Studies in Software Engineering , 2008, PPIG.

[18]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[19]  Mythili Thirugnanam.  Quality Metrics Tool for Object Oriented Programming , 2010 .

[20]  Roberto da Silva Bigonha,et al.  Identifying thresholds for object-oriented software metrics , 2012, J. Syst. Softw..

[21]  Praveen Ranjan Srivastava,et al.  Quantification of Software Quality Parameters Using Fuzzy Multi Criteria Approach , 2011, 2011 International Conference on Process Automation, Control and Computing.

[22]  Mauricio Finavaro Aniche,et al.  SATT: Tailoring Code Metric Thresholds for Different Software Architectures , 2016, SCAM.

[23]  Kazi Zakia Sultana,et al.  A Comparison of Nano-Patterns vs. Software Metrics in Vulnerability Prediction , 2018, 2018 25th Asia-Pacific Software Engineering Conference (APSEC).

[24]  Jianli Dong.  An Improved Fuzzy Synthesis Evaluation Algorithm for Software Quality , 2009, 2009 International Conference on Information Management, Innovation Management and Industrial Engineering.

[25]  D. Goodin.  The cambridge dictionary of statistics , 1999 .

[26]  Dur Abuzaid,et al.  The Visualization of Software Quality Metrics - A Systematic Literature Review , 2015 .

[27]  Pearl Brereton,et al.  Systematic literature reviews in software engineering - A systematic literature review , 2009, Inf. Softw. Technol..

[28]  Thomas Ruhroth,et al.  Measure, Diagnose, Refactor: A Formal Quality Cycle for Software Models , 2009, 2009 35th Euromicro Conference on Software Engineering and Advanced Applications.

[29]  Reiner R. Dumke,et al.  The 3C approach for Agile Quality Assurance , 2012, 2012 3rd International Workshop on Emerging Trends in Software Metrics (WETSoM).

[30]  Liang Ping.  A Quantitative Approach to Software Maintainability Prediction , 2010, 2010 International Forum on Information Technology and Applications.

[31]  Santosh Singh Rathore,et al.  Coupling and Cohesion Metrics for Object-Oriented Software: A Systematic Mapping Study , 2018, ISEC.

[32]  Heiko Koziolek,et al.  Quantifying software architecture quality report on the first international workshop on software architecture metrics , 2014, SOEN.

[33]  Kent Johnson,et al.  Measurement within the CMMi , 2008 .

[34]  Zoran Budimac,et al.  Towards a 'Universal' Software Metrics Tool - Motivation, Process and a Prototype , 2010, ICSOFT.

[35]  Peng Liang,et al.  An empirical investigation of modularity metrics for indicating architectural technical debt , 2014, QoSA '14.

[36]  Mukesh Bansal,et al.  Critical Analysis of Object Oriented Metrics in Software Development , 2014, 2014 Fourth International Conference on Advanced Computing & Communication Technologies.

[37]  Narayanan Kulathuramaiyer,et al.  Towards Resolving Software Quality-in-Use Measurement Challenges , 2015, ArXiv.

[38]  Anjana Gosain,et al.  Quality metrics for conceptual models for data warehouse focusing on dimension hierarchies , 2011, SOEN.

[39]  Lucas Batista Leite de Souza,et al.  Do software categories impact coupling metrics? , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[40]  Daniela Cruzes,et al.  Recommended Steps for Thematic Synthesis in Software Engineering , 2011, 2011 International Symposium on Empirical Software Engineering and Measurement.

[41]  Adilson Marques da Cunha,et al.  Software Product Measurement and Analysis in a Continuous Integration Environment , 2010, 2010 Seventh International Conference on Information Technology: New Generations.

[42]  T. Latha Maheswari,et al.  An effective rank approach to software defect prediction using software metrics , 2016, 2016 10th International Conference on Intelligent Systems and Control (ISCO).

[43]  Ruchika Malhotra,et al.  A new metric for predicting software change using gene expression programming , 2014, WETSoM 2014.

[44]  Peter Deussen,et al.  Current and Future Challenges of Software Engineering for Services and Applications , 2016, Cloud Forward.

[45]  Shari Lawrence Pfleeger,et al.  Software Metrics : A Rigorous and Practical Approach , 1998 .

[46]  Ruchika Malhotra,et al.  An Empirical Study to Redefine the Relationship between Software Design Metrics and Maintainability in High Data Intensive Applications , 2013 .

[47]  Thirumalaiselvi.  A Report on the Analysis of Metrics and Measures on Software Quality Factors – A Literature Study , 2014 .

[48]  Elisabetta Ronchieri,et al.  A Preliminary Mapping Study of Software Metrics Thresholds , 2016, ICSOFT-EA.

[49]  Heitor Augustus Xavier Costa,et al.  Metrics and statistical techniques used to evaluate internal quality of object-oriented software: A systematic mapping , 2016, 2016 35th International Conference of the Chilean Computer Science Society (SCCC).

[50]  Alan Oxley,et al.  A Coupling and Cohesion Metrics Suite for Object-Oriented Software , 2009, 2009 International Conference on Computer Technology and Development.

[51]  Marco Tulio Valente,et al.  RTTool: A Tool for Extracting Relative Thresholds for Source Code Metrics , 2014, 2014 IEEE International Conference on Software Maintenance and Evolution.

[52]  Parvinder S. Sandhu,et al.  A model for early prediction of faults in software systems , 2010, 2010 The 2nd International Conference on Computer and Automation Engineering (ICCAE).

[53]  Duncan Dubugras Alcoba Ruiz,et al.  SPDW+: a seamless approach for capturing quality metrics in software development environments , 2010, Software Quality Journal.

[54]  S. M. K. Quadri,et al.  Metrics, models and measurements in software reliability , 2012, 2012 IEEE 10th International Symposium on Applied Machine Intelligence and Informatics (SAMI).

[55]  Jeannette M. Wing,et al.  An Attack Surface Metric , 2011, IEEE Transactions on Software Engineering.

[56]  W. Duncan.  A GUIDE TO THE PROJECT MANAGEMENT BODY OF KNOWLEDGE , 1996 .

[57]  Sunita Chulani,et al.  Implementing quality metrics and goals at the corporate level , 2011, MSR '11.

[58]  Pooja,et al.  PREDICTION OF SOFTWARE DEFECTS USING OBJECT-ORIENTED METRICS , 2018 .

[59]  Feza Buzluca,et al.  E-Quality: A graph based object oriented software quality visualization tool , 2011, 2011 6th International Workshop on Visualizing Software for Understanding and Analysis (VISSOFT).

[60]  Kenji Takahashi,et al.  Metric-Based Quality Evaluations for Iterative Software Development Approaches Like Agile , 2012, 2012 IEEE 23rd International Symposium on Software Reliability Engineering Workshops.

[61]  Jing Chen,et al.  Software Maintainability Metrics Based on the Index System and Fuzzy Method , 2009, 2009 First International Conference on Information Science and Engineering.

[62]  Khabib Mustofa,et al.  A Systematic Mapping Review of Software Quality Measurement: Research Trends, Model, and Method , 2017 .

[63]  Arthur-Jozsef Molnar,et al.  Longitudinal Evaluation of Software Quality Metrics in Open-Source Applications , 2019, ENASE.

[64]  Sandeep Srivastava,et al.  Indirect method to measure software quality using CK-OO suite , 2013, 2013 International Conference on Intelligent Systems and Signal Processing (ISSP).

[65]  Abdul Azim Abdul Ghani,et al.  A mapping study to investigate component-based software system metrics , 2013, J. Syst. Softw..

[66]  Siffat Ullah Khan,et al.  Swot Analysis Of Software Quality Metrics For Global Software Development: A Systematic Literature Review Protocol , 2012 .

[67]  Sinan Eski,et al.  An Empirical Study on Object-Oriented Metrics and Software Evolution in Order to Reduce Testing Costs by Predicting Change-Prone Classes , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[68]  Ali Idri,et al.  Predicting Software Product Quality: A Systematic Mapping Study , 2015, Computación y Sistemas.

[69]  Amjed Tahir,et al.  A systematic mapping study on dynamic metrics and software quality , 2012, 2012 28th IEEE International Conference on Software Maintenance (ICSM).

[70]  Onur Demirörs,et al.  Towards the Development of a Defect Detection Tool for COSMIC Functional Size Measurement , 2013, 2013 Joint Conference of the 23rd International Workshop on Software Measurement and the 8th International Conference on Software Process and Product Measurement.

[71]  Carolyn B. Seaman,et al.  Gauging acceptance of software metrics: Comparing perspectives of managers and developers , 2009, 2009 3rd International Symposium on Empirical Software Engineering and Measurement.

[73]  Luigi Carro,et al.  Software Quality Metrics and their Impact on Embedded Software , 2008, 2008 5th International Workshop on Model-based Methodologies for Pervasive and Embedded Software.

[74]  Ming Li,et al.  Metric-based software reliability prediction approach and its application , 2017, Empirical Software Engineering.

[75]  Kapil Sharma,et al.  Improving software quality based on relationship among the change proneness and object oriented metrics , 2015, 2015 2nd International Conference on Computing for Sustainable Global Development (INDIACom).

[76]  T.R.Gopalakrishnan Nair,et al.  Estimation of Characteristics of a Software Team for Implementing Effective Inspection Process through Inspection Performance Metric , 2011 .

[77]  Amjed Tahir,et al.  An AOP-Based Approach for Collecting Software Maintainability Dynamic Metrics , 2010, 2010 Second International Conference on Computer Research and Development.

[78]  Taghi M. Khoshgoftaar,et al.  Choosing software metrics for defect prediction: an investigation on feature selection techniques , 2011, Softw. Pract. Exp..

[79]  Fabio Kon,et al.  A Study of the Relationships between Source Code Metrics and Attractiveness in Free Software Projects , 2010, 2010 Brazilian Symposium on Software Engineering.

[80]  Ana Regina Cavalcanti da Rocha,et al.  Establishing a Well-Founded Conceptualization about Software Measurement in High Maturity Levels , 2010, 2010 Seventh International Conference on the Quality of Information and Communications Technology.

[81]  Carlos E. Otero,et al.  A Quality-Based Requirement Prioritization Framework Using Binary Inputs , 2010, 2010 Fourth Asia International Conference on Mathematical/Analytical Modelling and Computer Simulation.

[82]  Klaus Lochmann,et al.  Integrating quality models and static analysis for comprehensive quality assessment , 2011, WETSoM '11.

[83]  T.C. Lethbridge,et al.  Guide to the Software Engineering Body of Knowledge (SWEBOK) and the Software Engineering Education Knowledge (SEEK) - a preliminary mapping , 2001, 10th International Workshop on Software Technology and Engineering Practice.

[84]  Pearl Brereton,et al.  Using mapping studies as the basis for further research - A participant-observer case study , 2011, Inf. Softw. Technol..

[85]  Georgios Lajios.  Software Metrics Suites for Project Landscapes , 2009, 2009 13th European Conference on Software Maintenance and Reengineering.

[86]  Aziz Deraman,et al.  Measuring the Unmeasurable Characteristics of Software Product Quality , 2010, Int. J. Adv. Comp. Techn..

[87]  Hany H. Ammar,et al.  Object oriented design metrics and tools a survey , 2010, 2010 The 7th International Conference on Informatics and Systems (INFOS).

[88]  Reinhold Plösch,et al.  Measuring, Assessing and Improving Software Quality based on Object-Oriented Design Principles , 2016, Open Comput. Sci..

[89]  M. K. Debbarma,et al.  Static and dynamic software metrics complexity analysis in regression testing , 2012, 2012 International Conference on Computer Communication and Informatics.

[90]  Sanjay Misra,et al.  A Suite of Object Oriented Cognitive Complexity Metrics , 2018, IEEE Access.

[91]  Pearl Brereton,et al.  The educational value of mapping studies of software engineering literature , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[92]  Ömer Faruk Arar,et al.  Deriving thresholds of software metrics to predict faults on open source software: Replicated case studies , 2016, Expert Syst. Appl..

[93]  P. K. Kapur,et al.  Measuring software testing efficiency using two-way assessment technique , 2014, Proceedings of 3rd International Conference on Reliability, Infocom Technologies and Optimization.

[94]  Mariza A. S. Bigonha,et al.  An exploratory study on cooccurrence of design patterns and bad smells using software metrics , 2019, Softw. Pract. Exp..

[95]  Sven Apel,et al.  Exploring Software Measures to Assess Program Comprehension , 2011, 2011 International Symposium on Empirical Software Engineering and Measurement.

[96]  Kalpana Sharma,et al.  Proposed T-Model to cover 4S quality metrics based on empirical study of root cause of software failures , 2019 .

[97]  Alok Mishra,et al.  DevOps and software quality: A systematic mapping , 2020, Comput. Sci. Rev..

[98]  Edna Dias Canedo,et al.  Construction of a Software Measurement Tool Using Systematic Literature Review , 2018, 2018 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData).

[99]  Chitra Babu,et al.  Evaluation of reusability in Aspect Oriented Software using inheritance metrics , 2014, 2014 IEEE International Conference on Advanced Communications, Control and Computing Technologies.

[100]  Ahmed Ali Abdalla Esmin,et al.  Applying Swarm Ensemble Clustering Technique for Fault Prediction Using Software Metrics , 2014, 2014 13th International Conference on Machine Learning and Applications.

[101]  Y. Raghu Reddy,et al.  Software Quality Models: A Systematic Mapping Study , 2019, 2019 IEEE/ACM International Conference on Software and System Processes (ICSSP).

[102]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.