Software quality assessment model: a systematic mapping study

Quality model is regarded as a well-accepted approach for assessing, managing and improving software product quality. There are three categories of quality models for software products, i.e., definition model, assessment model, and prediction model. Quality assessment model (QAM) is a metric-based approach to assess the software quality. It is typically regarded as of high importance for its clear method on how to assess a system. However, the current state-of-the-art in QAM research is under limited investigation. To address this gap, the paper provides an organized and synthesized summary of the current QAMs. In detail, we conduct a systematic mapping study (SMS) for structuring the relevant articles. We obtain a total of 716 papers from the five databases, and 31 papers are selected as relevant studies at last. In summary, our work focuses on QAMs from the following aspects: software metrics, quality factors, aggregation methods, evaluation methods and tool support. According to the analysis results, our work discovers five needs that researchers in this area should continue to address: (1) new method and criteria to tailor a quality framework (i.e., structure of software metrics and quality factors) according to different specifics, (2) systematic investigations on the effectiveness, strength and weakness of different aggregation methods to guide the method selection in different context, (3) more investigations on evaluating QAMs in the context of industrial cases, (4) further investigations or real-world case studies on the QAMs related tools, and (5) building a public and diverse software benchmark which can be adopted in different application context.

[1]  Andy Zaidman,et al.  Test Code Quality and Its Relation to Issue Handling Performance , 2014, IEEE Transactions on Software Engineering.

[2]  Kristoffer Kvam,et al.  Legacy system exorcism by Pareto's principle , 2005, OOPSLA '05.

[3]  Magne Jørgensen,et al.  A Systematic Review of Software Development Cost Estimation Studies , 2007, IEEE Transactions on Software Engineering.

[4]  Xavier Franch,et al.  Using Quality Models in Software Package Selection , 2003, IEEE Softw..

[5]  Laurence Tianruo Yang,et al.  Fuzzy Logic with Engineering Applications , 1999 .

[6]  Joost Visser,et al.  Standardized code quality benchmarking for improving software maintainability , 2011, Software Quality Journal.

[7]  Ioannis Stamelos,et al.  The SQO-OSS Quality Model: Measurement Based Open Source Software Evaluation , 2008, OSS.

[8]  Ewan D. Tempero,et al.  A systematic review of software maintainability prediction and metrics , 2009, 2009 3rd International Symposium on Empirical Software Engineering and Measurement.

[9]  Ling Xu,et al.  A Systematic Mapping Study of Quality Assessment Models for Software Products , 2017, 2017 International Conference on Software Analysis, Testing and Evolution (SATE).

[10]  Sarah Smith Heckman,et al.  A systematic literature review of actionable alert identification techniques for automated static code analysis , 2011, Inf. Softw. Technol..

[11]  Ünal Çavusoglu,et al.  A novel defect prediction method for web pages using k-means++ , 2015, Expert Syst. Appl..

[12]  Arne-Kristian Groven,et al.  Quality Models for Free/Libre Open Source Software Towards the “Silver Bullet”? , 2010, 2010 36th EUROMICRO Conference on Software Engineering and Advanced Applications.

[13]  Reinhold Plösch,et al.  A Comprehensive Code-Based Quality Model for Embedded Systems: Systematic Development and Validation by Industrial Projects , 2012, 2012 IEEE 23rd International Symposium on Software Reliability Engineering.

[14]  Nicolas Anquetil,et al.  Software quality metrics aggregation in industry , 2013, J. Softw. Evol. Process..

[15]  Sandeep Srivastava,et al.  Indirect method to measure software quality using CK-OO suite , 2013, 2013 International Conference on Intelligent Systems and Signal Processing (ISSP).

[16]  D. V. Radhika,et al.  SCQAM: a scalable structured code quality assessment method for industrial software , 2014, ICPC 2014.

[17]  Pearl Brereton,et al.  Lessons from applying the systematic literature review process within the software engineering domain , 2007, J. Syst. Softw..

[18]  Reinhold Plösch,et al.  Operationalised product quality models and assessment: The Quamoco approach , 2014, Inf. Softw. Technol..

[19]  R. Geoff Dromey,et al.  A Model for Software Product Quality , 1995, IEEE Trans. Software Eng..

[20]  Alessandro Marchetto OQMw: An OO Quality Model for Web Applications , 2009 .

[21]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[22]  Joost Visser,et al.  A survey-based study of the mapping of system properties to ISO/IEC 9126 maintainability characteristics , 2009, 2009 IEEE International Conference on Software Maintenance.

[23]  Robert C. Martin Agile Software Development, Principles, Patterns, and Practices , 2002 .

[24]  Hui Gao,et al.  2-D Software Quality Model and Case Study in Software Flexibility Research , 2008, 2008 International Conference on Computational Intelligence for Modelling Control & Automation.

[25]  Reinhold Plösch,et al.  On the Validity of the IT-CISQ Quality Model for Automatic Measurement of Maintainability , 2015, 2015 IEEE 39th Annual Computer Software and Applications Conference.

[26]  Ioannis Stamelos,et al.  Software Product and Process Assessment through Profile-Based Evaluation , 2003, Int. J. Softw. Eng. Knowl. Eng..

[27]  Mauricio A. Saca Refactoring improving the design of existing code , 2017, 2017 IEEE 37th Central America and Panama Convention (CONCAPAN XXXVII).

[28]  Bruce E. Barrett,et al.  Decision quality using ranked attribute weights , 1996 .

[29]  Reinhold Plösch,et al.  Objective safety compliance checks for source code , 2014, ICSE Companion.

[30]  Richard T. Watson,et al.  Analyzing the Past to Prepare for the Future: Writing a Literature Review , 2002, MIS Q..

[31]  Alexis Tsoukiàs,et al.  Multi-Criteria Methodology Contribution to the Software Quality Evaluation , 2001, Software Quality Journal.

[32]  Maurice H. Halstead,et al.  Elements of software science , 1977 .

[33]  Reinhold Plösch,et al.  The Quamoco product quality modelling and assessment approach , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[34]  Reinhold Plösch,et al.  The EMISQ method and its tool support-expert-based evaluation of internal software quality , 2008, Innovations in Systems and Software Engineering.

[35]  Barry Boehm,et al.  Characteristics of software quality , 1978 .

[36]  A. I. Chernyi The ISI Web of Knowledge, a modern system for the information support of scientific research: a review , 2009, Scientific and Technical Information Processing.

[37]  Ruchika Malhotra,et al.  A systematic review of machine learning techniques for software fault prediction , 2015, Appl. Soft Comput..

[38]  Richard Mark Soley,et al.  The Consortium for IT Software Quality , 2010, SEAFOOD.

[39]  Stéphane Ducasse,et al.  The squale model — A practice-based industrial quality model , 2009, 2009 IEEE International Conference on Software Maintenance.

[40]  Tibor Gyimóthy,et al.  A probabilistic software quality model , 2011, 2011 27th IEEE International Conference on Software Maintenance (ICSM).

[41]  Barbara Kitchenham,et al.  What's up with software metrics? - A preliminary mapping study , 2010, J. Syst. Softw..

[42]  Magne Jørgensen,et al.  A Systematic Review of Software Development Cost Estimation Studies , 2007 .

[43]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[44]  Teresita Rojas,et al.  Construction of a Systemic Quality Model for Evaluating a Software Product , 2003, Software Quality Journal.

[45]  Yanfang Liu,et al.  Evaluate the Quality of Foundational Software Platform by Bayesian Network , 2010, AICI.

[46]  R. Dromey,et al.  A Model for Software Product Quality , 1995, IEEE Trans. Software Eng..

[47]  Coral Calero,et al.  A Systematic Mapping Study of Software Reliability Modeling , 2014, Inf. Softw. Technol..

[48]  Reinhold Plösch,et al.  Objective Measurement of Safety in the Context of IEC 61508-3 , 2013, 2013 39th Euromicro Conference on Software Engineering and Advanced Applications.

[49]  Foutse Khomh,et al.  DEQUALITE: building design-based software quality models , 2008 .

[50]  Stefan Wagner,et al.  Software Product Quality Control , 2013, Springer Berlin Heidelberg.

[51]  María José Escalona Cuaresma,et al.  Open source tools for measuring the Internal Quality of Java software products. A survey , 2013, Comput. Stand. Interfaces.

[52]  Oleg I. Larichev,et al.  An approach to ordinal classification problems , 1994 .

[53]  Jürgen Münch,et al.  CQML Scheme: A Classification Scheme for Comprehensive Quality Model Landscapes , 2009, 2009 35th Euromicro Conference on Software Engineering and Advanced Applications.

[54]  J. M. Singer,et al.  IEEE Recommended Practice for Software Requirements SpeciÞcations , 1993 .

[55]  Silvia Mara Abrahão,et al.  A systematic review of quality attributes and measures for software product lines , 2011, Software Quality Journal.

[56]  Girish Suryanarayana,et al.  MIDAS: A design quality assessment method for industrial software , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[57]  Dietmar P. F. Möller,et al.  Systems and Software Engineering , 2016 .

[58]  Carl G. Davis,et al.  A Hierarchical Model for Object-Oriented Design Quality Assessment , 2002, IEEE Trans. Software Eng..

[59]  Lida Xu,et al.  CLOUDQUAL: A Quality Model for Cloud Services , 2014, IEEE Transactions on Industrial Informatics.

[60]  Mario Piattini,et al.  A systematic review measurement in software engineering: State-of-the-art in Measures , 2006, ICSOFT.

[61]  Banu Diri,et al.  A systematic review of software fault prediction studies , 2009, Expert Syst. Appl..

[62]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[63]  Murat Köksalan,et al.  An interactive approach for placing alternatives in preference classes , 2003, Eur. J. Oper. Res..

[64]  Thierry Coq,et al.  The SQALE Analysis Model: An Analysis Model Compliant with the Representation Condition for Assessing the Quality of Software Source Code , 2010, 2010 Second International Conference on Advances in System Testing and Validation Lifecycle.

[65]  Jean-Francois Girard,et al.  An Activity-Based Quality Model for Maintainability , 2007, 2007 IEEE International Conference on Software Maintenance.

[66]  Kai Petersen,et al.  Systematic Mapping Studies in Software Engineering , 2008, EASE.

[67]  Chonlameth Arpnikanondt,et al.  A Visualization Technique for Metrics-Based Hierarchical Quality Models , 2012, 2012 19th Asia-Pacific Software Engineering Conference.

[68]  Witold Pedrycz,et al.  Software quality measurement: concepts and fuzzy neural relational model , 1998, 1998 IEEE International Conference on Fuzzy Systems Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36228).

[69]  Ajit Pratap Singh,et al.  Integrated Software Quality Evaluation: A Fuzzy Multi-Criteria Approach , 2011, J. Inf. Process. Syst..

[70]  Yani Widyani,et al.  Efficiency measurement of Java Android code , 2014, 2014 International Conference on Data and Software Engineering (ICODSE).

[71]  Richard Mark Soley,et al.  The Consortium for IT Software Quality (CISQ) , 2013, SWQD.

[72]  Elmar Jürgens,et al.  Software quality models: Purposes, usage scenarios and requirements , 2009, 2009 ICSE Workshop on Software Quality.