A Systematic Mapping Study of Quality Assessment Models for Software Products

Quality model is regarded as a well-accepted approach for assessing, managing and improving software product quality. There are three categories of quality models for software products, i.e., definition model, assessment model, and prediction model. Quality assessment model (QAM) is a metric-based approach to assess the software quality. It is typically regarded as of high importance for its clear method on how to assess a system. However, the current state-of-the-art in QAM research is under limited investigation. To address this gap, the paper provides an organized and synthesized summary of the current QAMs. In detail, we conduct a systematic mapping study (SMS) for structuring the relevant articles. We obtain a total of 716 papers from the five databases, and 31 papers are selected as relevant studies at last. In summary, our work focuses on QAMs from the following aspects: software metrics, quality factors, evaluation methods and tool support.

[1]  Jürgen Münch,et al.  CQML Scheme: A Classification Scheme for Comprehensive Quality Model Landscapes , 2009, 2009 35th Euromicro Conference on Software Engineering and Advanced Applications.

[2]  Arne-Kristian Groven,et al.  Quality Models for Free/Libre Open Source Software Towards the “Silver Bullet”? , 2010, 2010 36th EUROMICRO Conference on Software Engineering and Advanced Applications.

[3]  Deepak Goyal,et al.  A hierarchical model for object-oriented design quality assessment , 2015 .

[4]  Silvia Mara Abrahão,et al.  A systematic review of quality attributes and measures for software product lines , 2011, Software Quality Journal.

[5]  Girish Suryanarayana,et al.  MIDAS: A design quality assessment method for industrial software , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[6]  Coral Calero,et al.  A Systematic Mapping Study of Software Reliability Modeling , 2014, Inf. Softw. Technol..

[7]  Reinhold Plösch,et al.  A Comprehensive Code-Based Quality Model for Embedded Systems: Systematic Development and Validation by Industrial Projects , 2012, 2012 IEEE 23rd International Symposium on Software Reliability Engineering.

[8]  Reinhold Plösch,et al.  Objective Measurement of Safety in the Context of IEC 61508-3 , 2013, 2013 39th Euromicro Conference on Software Engineering and Advanced Applications.

[9]  Foutse Khomh,et al.  DEQUALITE: building design-based software quality models , 2008 .

[10]  Reinhold Plösch,et al.  Operationalised product quality models and assessment: The Quamoco approach , 2014, Inf. Softw. Technol..

[11]  Maurice H. Halstead,et al.  Elements of software science , 1977 .

[12]  Sandeep Srivastava,et al.  Indirect method to measure software quality using CK-OO suite , 2013, 2013 International Conference on Intelligent Systems and Signal Processing (ISSP).

[13]  Pearl Brereton,et al.  Lessons from applying the systematic literature review process within the software engineering domain , 2007, J. Syst. Softw..

[14]  Ling Xu,et al.  Learning to Aggregate: An Automated Aggregation Method for Software Quality Model , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C).

[15]  Stefan Wagner,et al.  Software Product Quality Control , 2013, Springer Berlin Heidelberg.

[16]  Reinhold Plösch,et al.  The Quamoco product quality modelling and assessment approach , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[17]  R. Geoff Dromey,et al.  A Model for Software Product Quality , 1995, IEEE Trans. Software Eng..

[18]  Yani Widyani,et al.  Efficiency measurement of Java Android code , 2014, 2014 International Conference on Data and Software Engineering (ICODSE).

[19]  Magne Jørgensen,et al.  A Systematic Review of Software Development Cost Estimation Studies , 2007 .

[20]  Elmar Jürgens,et al.  Software quality models: Purposes, usage scenarios and requirements , 2009, 2009 ICSE Workshop on Software Quality.

[21]  Alessandro Marchetto OQMw: An OO Quality Model for Web Applications , 2009 .

[22]  Reinhold Plösch,et al.  Objective safety compliance checks for source code , 2014, ICSE Companion.

[23]  Ling Xu,et al.  Automating Aggregation for Software Quality Modeling , 2017, 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[24]  Ling Xu,et al.  Automatically classifying software changes via discriminative topic model: Supporting multi-category and cross-project , 2016, J. Syst. Softw..

[25]  A. I. Chernyi The ISI Web of Knowledge, a modern system for the information support of scientific research: a review , 2009, Scientific and Technical Information Processing.

[26]  Ruchika Malhotra,et al.  A systematic review of machine learning techniques for software fault prediction , 2015, Appl. Soft Comput..

[27]  Richard Mark Soley,et al.  The Consortium for IT Software Quality , 2010, SEAFOOD.

[28]  Alexis Tsoukiàs,et al.  Multi-Criteria Methodology Contribution to the Software Quality Evaluation , 2001, Software Quality Journal.

[29]  Stéphane Ducasse,et al.  The squale model — A practice-based industrial quality model , 2009, 2009 IEEE International Conference on Software Maintenance.

[30]  Andy Zaidman,et al.  Test Code Quality and Its Relation to Issue Handling Performance , 2014, IEEE Transactions on Software Engineering.

[31]  양해술,et al.  ISO/IEC 14598-3을 적용한 개발자의 제품평가 방법 , 1999 .

[32]  Kristoffer Kvam,et al.  Legacy system exorcism by Pareto's principle , 2005, OOPSLA '05.

[33]  Anna Rita Fasolino,et al.  Lo Standard ISO/IEC 9126 – Software engineering – Product Quality , 2010 .

[34]  Xavier Franch,et al.  Using Quality Models in Software Package Selection , 2003, IEEE Softw..

[35]  Ioannis Stamelos,et al.  The SQO-OSS Quality Model: Measurement Based Open Source Software Evaluation , 2008, OSS.

[36]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[37]  D. V. Radhika,et al.  SCQAM: a scalable structured code quality assessment method for industrial software , 2014, ICPC 2014.

[38]  Barbara Kitchenham,et al.  What's up with software metrics? - A preliminary mapping study , 2010, J. Syst. Softw..

[39]  Richard T. Watson,et al.  Analyzing the Past to Prepare for the Future: Writing a Literature Review , 2002, MIS Q..

[40]  María José Escalona Cuaresma,et al.  Open source tools for measuring the Internal Quality of Java software products. A survey , 2013, Comput. Stand. Interfaces.

[41]  Barry Boehm,et al.  Characteristics of software quality , 1978 .

[42]  Yanfang Liu,et al.  Evaluate the Quality of Foundational Software Platform by Bayesian Network , 2010, AICI.

[43]  Magne Jørgensen,et al.  A Systematic Review of Software Development Cost Estimation Studies , 2007, IEEE Transactions on Software Engineering.

[44]  Lida Xu,et al.  CLOUDQUAL: A Quality Model for Cloud Services , 2014, IEEE Transactions on Industrial Informatics.

[45]  Reinhold Plösch,et al.  The EMISQ Method - Expert Based Evaluation of Internal Software Quality , 2007, 31st IEEE Software Engineering Workshop (SEW 2007).

[46]  Joost Visser,et al.  Standardized code quality benchmarking for improving software maintainability , 2011, Software Quality Journal.

[47]  Ünal Çavusoglu,et al.  A novel defect prediction method for web pages using k-means++ , 2015, Expert Syst. Appl..

[48]  Tibor Gyimóthy,et al.  A probabilistic software quality model , 2011, 2011 27th IEEE International Conference on Software Maintenance (ICSM).

[49]  David Lo,et al.  File-Level Defect Prediction: Unsupervised vs. Supervised Models , 2017, 2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM).

[50]  Thierry Coq,et al.  The SQALE Analysis Model: An Analysis Model Compliant with the Representation Condition for Assessing the Quality of Software Source Code , 2010, 2010 Second International Conference on Advances in System Testing and Validation Lifecycle.

[51]  Mauricio A. Saca Refactoring improving the design of existing code , 2017, 2017 IEEE 37th Central America and Panama Convention (CONCAPAN XXXVII).

[52]  Hui Gao,et al.  2-D Software Quality Model and Case Study in Software Flexibility Research , 2008, 2008 International Conference on Computational Intelligence for Modelling Control & Automation.

[53]  Jean-Francois Girard,et al.  An Activity-Based Quality Model for Maintainability , 2007, 2007 IEEE International Conference on Software Maintenance.

[54]  Kai Petersen,et al.  Systematic Mapping Studies in Software Engineering , 2008, EASE.

[55]  Chonlameth Arpnikanondt,et al.  A Visualization Technique for Metrics-Based Hierarchical Quality Models , 2012, 2012 19th Asia-Pacific Software Engineering Conference.

[56]  Witold Pedrycz,et al.  Software quality measurement: concepts and fuzzy neural relational model , 1998, 1998 IEEE International Conference on Fuzzy Systems Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36228).

[57]  Ajit Pratap Singh,et al.  Integrated Software Quality Evaluation: A Fuzzy Multi-Criteria Approach , 2011, J. Inf. Process. Syst..

[58]  Teresita Rojas,et al.  Construction of a Systemic Quality Model for Evaluating a Software Product , 2003, Software Quality Journal.

[59]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[60]  Reinhold Plösch,et al.  On the Validity of the IT-CISQ Quality Model for Automatic Measurement of Maintainability , 2015, 2015 IEEE 39th Annual Computer Software and Applications Conference.

[61]  Ioannis Stamelos,et al.  Software Product and Process Assessment through Profile-Based Evaluation , 2003, Int. J. Softw. Eng. Knowl. Eng..

[62]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[63]  Mehwish Riaz,et al.  A systematic review of software maintainability prediction and metrics , 2009, ESEM 2009.

[64]  Mario Piattini,et al.  A systematic review measurement in software engineering: State-of-the-art in Measures , 2006, ICSOFT.

[65]  Banu Diri,et al.  A systematic review of software fault prediction studies , 2009, Expert Syst. Appl..

[66]  Robert J. Winter Cpt Agile Software Development: Principles, Patterns, and Practices , 2014 .