A SQuaRE-based software quality evaluation framework and its case study

Software stakeholders, including developers, managers, and end users, require high quality software products. Several works have aimed to identify software quality, but the quality of software products is often not comprehensively, specifically, or effectively defined because previous approaches have focused on certain quality aspects. Moreover, the evaluation results of quality metrics often depend on software stakeholders so that it is often hard to compare quality evaluation results across software products. ISO/IEC has tried to define evaluation methods for the quality of software products and provide common standards, called the SQuaRE (Systems and software Quality Requirements and Evaluation) series including ISO/IEC 25022:2016 and ISO/IEC 25023:2016. However, the SQuaRE series include ambiguous metrics so that it is not always easy to apply the series to products and compare results. In this paper, we propose a SQuaRE-based software quality evaluation framework, which successfully concretized many product metrics and quality in use metrics originally defined in the SQuaRE series1. Through a case study targeting a commercial software product, we confirmed that our framework is concretely applicable to the software package/service product.

[1]  Hironori Washizaki,et al.  A Framework for Measuring and Evaluating Program Source Code Quality , 2007, PROFES.

[2]  F. Reichheld.  The one number you need to grow. , 2003, Harvard business review.

[3]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[4]  Hironori Washizaki,et al.  Initial Framework for Software Quality Evaluation Based on ISO/IEC 25022 and ISO/IEC 25023 , 2016, 2016 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C).

[5]  Hironori Washizaki,et al.  Quality evaluation of embedded software in robot software design contest , 2007 .

[6]  Hironori Washizaki,et al.  A Coupling-based Complexity Metric for Remote Component-based Software Systems Toward Maintainability Estimation , 2006, 2006 13th Asia Pacific Software Engineering Conference (APSEC'06).

[7]  Atsuhiro Takasu,et al.  Extracting Relations among Embedded Software Design Patterns , 2005, Trans. SDPS.

[8]  Hironori Washizaki,et al.  A Metrics Suite for Measuring Quality Characteristics of JavaBeans Components , 2008, PROFES.

[9]  Atsuhiro Takasu,et al.  Relation Analysis Among Patterns on Software Development Process , 2005, PROFES.

[10]  Mary Corbett,et al.  SUMI: the Software Usability Measurement Inventory , 1993, Br. J. Educ. Technol..

[11]  B. Boehm,et al.  Modeling Software Defect Introduction and Removal : COQUALMO ( COnstructive QUALity MOdel ) , 1999 .

[12]  Colin G. Drury,et al.  Foundations for an Empirically Determined Scale of Trust in Automated Systems , 2000 .

[13]  Nagoya,et al.  Proceedings 1st International Workshop on Software Patterns and Quality (spaqu'07) Proceedings 1st International Workshop on Software Patterns and Quality (spaqu'07) Workshop Organizers (program Co-chairs) Program Committee Linda Rising, Independent Consultant External Reviewers International Worksh , 2022 .

[14]  Hironori Washizaki,et al.  Experiments on quality evaluation of embedded software in Japan robot software design contest , 2006, ICSE.

[15]  Bashar Nuseibeh,et al.  Security patterns: comparing modeling approaches , 2010 .

[16]  Eduardo B. Fernández,et al.  Classifying Security Patterns , 2008, APWeb.

[17]  Hironori Washizaki,et al.  A Generalized Software Reliability Model Considering Uncertainty and Dynamics in Development , 2013, PROFES.

[18]  Stefan Wagner.  Software Quality Models in Practice Survey Results , 2012 .

[19]  Eduardo B. Fernández,et al.  Modeling Misuse Patterns , 2009, 2009 International Conference on Availability, Reliability and Security.

[20]  Frank Elberzhager,et al.  Managing software quality through a hybrid defect content and effectiveness model , 2008, ESEM '08.

[21]  Jürgen Münch,et al.  Software project control centers: concepts and approaches , 2004, J. Syst. Softw..

[22]  Jürgen Münch,et al.  CQML Scheme: A Classification Scheme for Comprehensive Quality Model Landscapes , 2009, 2009 35th Euromicro Conference on Software Engineering and Advanced Applications.

[23]  Hironori Washizaki,et al.  Open Code Coverage Framework: A Consistent and Flexible Framework for Measuring Test Coverage Supporting Multiple Programming Languages , 2010, 2010 10th International Conference on Quality Software.

[24]  Hiroyuki Tanabe,et al.  Reusability Metrics for Program Source Code Written in C Language and Their Evaluation , 2012, PROFES.

[25]  D. Watson,et al.  Development and validation of brief measures of positive and negative affect: the PANAS scales. , 1988, Journal of personality and social psychology.

[26]  Barbara A. Kitchenham,et al.  The use and usefulness of the ISO/IEC 9126 quality standard , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[27]  Hironori Washizaki,et al.  Abstract security patterns , 2008 .

[28]  Jürgen Münch,et al.  Model-based Product Quality Evaluation with Multi-Criteria Decision Analysis , 2014, ArXiv.

[29]  Alain Abran,et al.  Usability Meanings and Interpretations in ISO Standards , 2003, Software Quality Journal.

[30]  Michael Kläs,et al.  Model-Based Quality Management of Software Development Projects , 2014, Software Project Management in a Changing World.

[31]  Atsuhiro Takasu,et al.  Analyzing relations among software patterns based on document similarity , 2005, International Conference on Information Technology: Coding and Computing (ITCC'05) - Volume II.

[32]  Yasuyuki Tanaka,et al.  Continuous Product-Focused Project Monitoring with Trend Patterns and GQM , 2014, 2014 21st Asia-Pacific Software Engineering Conference.

[33]  Reinhold Plösch,et al.  The Quamoco product quality modelling and assessment approach , 2012, 2012 34th International Conference on Software Engineering (ICSE).