Software Product Quality Models, Developments, Trends, and Evaluation

Software product quality models have evolved in their abilities to capture and describe the abstract notion of software quality since the 1970s. Many models constructed deal with a specific part of software quality only which makes them ineligible to assess the quality of software products as a whole. Former publications failed to thoroughly examine and list all the available models which attempt to describe each known property of software product quality. This paper discovers such complete software product quality models published since 2000; moreover, it endeavours to measure the relevance of each model quantitatively by introducing indicators with regard to the scientific and industrial community. The identified 23 software product quality model classes differ significantly in terms of publication intensity, publication range, quality score average, relevance score and the 12-month average of the Google Relative Search Index. The results offer a foundation for selecting the appropriate software product quality model for use or for extension if newly identified quality properties need to be connected to a general context. Furthermore, the experiences accumulated on the field of software product quality modelling motivated researchers to successfully transfer the concepts to other areas where abstract entities need to be compared or assessed including the quality of higher educational teaching and business processes, which is also briefly highlighted in the paper.

[1]  Reinhold Plösch,et al.  Operationalised product quality models and assessment: The Quamoco approach , 2014, Inf. Softw. Technol..

[2]  Morgan Ericsson,et al.  Quality Models Inside Out: Interactive Visualization of Software Metrics by Means of Joint Probabilities , 2018, 2018 IEEE Working Conference on Software Visualization (VISSOFT).

[3]  Norhan Abd Rahman,et al.  Simulation of Flood Extent Mapping by InfoWorks RS-Case Study for Tropical Catchment , 2011 .

[4]  Markus Herrmannsdoerfer,et al.  The quamoco tool chain for quality modeling and assessment , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[5]  Souheil Khaddaj,et al.  Use of an adaptable quality model approach in a production support environment , 2009, J. Syst. Softw..

[6]  L. Zadeh Is there a need for fuzzy logic , 2008, NAFIPS 2008.

[7]  Richard Mark Soley,et al.  Quality concerns in large-scale and complex software-intensive systems , 2015 .

[8]  Iman Attarzadeh,et al.  A novel soft computing model to increase the accuracy of software development cost estimation , 2010, 2010 The 2nd International Conference on Computer and Automation Engineering (ICCAE).

[9]  Srinarayan Sharma,et al.  Impact of customization over software quality in ERP projects: an empirical study , 2016, Software Quality Journal.

[10]  R. Geoff Dromey,et al.  A Model for Software Product Quality , 1995, IEEE Trans. Software Eng..

[11]  Miroslav Bures,et al.  Aspects of Quality in Internet of Things (IoT) Solutions: A Systematic Mapping Study , 2019, IEEE Access.

[12]  Reinhold Plösch,et al.  Towards an Operational Safety Standard for Software: Modelling IEC 61508 Part 3 , 2011, 2011 18th IEEE International Conference and Workshops on Engineering of Computer-Based Systems.

[13]  Reinhold Plösch,et al.  A Method for Continuous Code Quality Management Using Static Analysis , 2010, 2010 Seventh International Conference on the Quality of Information and Communications Technology.

[14]  Stefan Wagner,et al.  Introduction of static quality analysis in small- and medium-sized software enterprises: experiences from technology transfer , 2013, Software Quality Journal.

[15]  A. Davila,et al.  Establishing software product quality requirements according to international standards , 2006, IEEE Latin America Transactions.

[16]  Robert B. Grady,et al.  Practical Software Metrics for Project Management and Process Improvement , 1992 .

[17]  V. Suma,et al.  Quality attribute focused multilayer requirement elicitation: Judicious approach to drive business value , 2013, 2013 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[18]  Hui Gao,et al.  2-D Software Quality Model and Case Study in Software Flexibility Research , 2008, 2008 International Conference on Computational Intelligence for Modelling Control & Automation.

[19]  Andreas S. Andreou,et al.  A quality framework for developing and evaluating original software components , 2007, Inf. Softw. Technol..

[20]  Taghi M. Khoshgoftaar,et al.  Fuzzy logic techniques for software reliability engineering , 2001 .

[21]  Thomas I. Strasser,et al.  Assessing the Integration of Software Agents and Industrial Automation Systems with ISO/IEC 25010 , 2018, 2018 IEEE 16th International Conference on Industrial Informatics (INDIN).

[22]  Tibor Gyimóthy,et al.  A cost model based on software maintainability , 2012, 2012 28th IEEE International Conference on Software Maintenance (ICSM).

[23]  Forrest Shull,et al.  Using the ISO/IEC 9126 product quality model to classify defects: A controlled experiment , 2012, EASE.

[24]  Ying-xing Li,et al.  A Fuzzy Comprehensive Quality Evaluation for the Digitizing Software of Ethnic Antiquarian Resources , 2008, 2008 International Conference on Computer Science and Software Engineering.

[25]  Mehrdad Sabetzadeh,et al.  Characterizing the Chain of Evidence for Software Safety Cases: A Conceptual Model Based on the IEC 61508 Standard , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[26]  Xavier Burgués Illa,et al.  A MOF-Compliant Approach to Software Quality Modeling , 2005, ER.

[27]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[28]  Robert B. Grady,et al.  Software Metrics: Establishing a Company-Wide Program , 1987 .

[29]  Eric Knauss,et al.  Assessing the Quality of Software Requirements Specifications , 2008, 2008 16th IEEE International Requirements Engineering Conference.

[30]  Claude Y. Laporte,et al.  Evolving a Corporate Software Quality Assessment Exercise: A Migration Path to ISO/IEC 9126 , 2004 .

[31]  Lotfi A. Zadeh,et al.  Fuzzy logic = computing with words , 1996, IEEE Trans. Fuzzy Syst..

[32]  Reinhold Plösch,et al.  The Quamoco product quality modelling and assessment approach , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[33]  Reinhold Plösch,et al.  The EMISQ method and its tool support-expert-based evaluation of internal software quality , 2008, Innovations in Systems and Software Engineering.

[34]  Zhenyu Liu,et al.  Agent-Based Online Quality Measurement Approach in Cloud Computing Environment , 2010, 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[35]  Joost Visser,et al.  Interpretation of Source Code Clusters in Terms of the ISO/IEC-9126 Maintainability Characteristics , 2008, 2008 12th European Conference on Software Maintenance and Reengineering.

[36]  Francisco Chiclana,et al.  Acta Polytechnica Hungarica Vol.11. No.3. (2014.) , 2014 .

[37]  Thierry Coq Verification and Validation in the Recommended Practice for Integrated Software-Dependent Systems , 2009, 2009 First International Conference on Advances in System Testing and Validation Lifecycle.

[38]  Suhaimi Ibrahim,et al.  A unified framework for software engineering process improvement — A taxonomy comparative analysis , 2011, 2011 Malaysian Conference in Software Engineering.

[39]  Meir M. Lehman,et al.  Rules and Tools for Software Evolution Planning and Management , 2001, Ann. Softw. Eng..

[40]  Onur Demirors,et al.  Application of process quality measurement frameworks for human resource management processes , 2011, 2011 IEEE International Conference on Quality and Reliability.

[41]  A. Selçuk Güceglioglu,et al.  A Process Based Model for Measuring Process Quality Attributes , 2005, EuroSPI.

[42]  Jingyue Li,et al.  Interface Information Management Tools for the Maritime and Oil and Gas Industry , 2015, 2015 IEEE 39th Annual Computer Software and Applications Conference.

[43]  T. Rajaram,et al.  Continual monitoring of code quality , 2011, ISEC.

[44]  Stéphane Ducasse,et al.  The squale model — A practice-based industrial quality model , 2009, 2009 IEEE International Conference on Software Maintenance.

[45]  Troy Martin Hughes SAS®Data Analytic Development: Dimensions of Software Quality , 2016 .

[46]  Pearl Brereton,et al.  A systematic review of systematic review process research in software engineering , 2013, Inf. Softw. Technol..

[47]  Ahmed Seffah,et al.  QUIM: a framework for quantifying usability metrics in software quality models , 2001, Proceedings Second Asia-Pacific Conference on Quality Software.

[48]  Tibor Gyimóthy,et al.  A probabilistic software quality model , 2011, 2011 27th IEEE International Conference on Software Maintenance (ICSM).

[49]  Fu Lee Wang,et al.  Network Computing and Information Security , 2012, Communications in Computer and Information Science.

[50]  Mark E. Borsuk,et al.  A software tool for translating deterministic model results into probabilistic assessments of water quality standard compliance , 2009, Environ. Model. Softw..

[51]  Jean-Louis Letouzey,et al.  Managing Technical Debt with the SQALE Method , 2012, IEEE Software.

[52]  Wei Hu,et al.  Quality model based on ISO/IEC 9126 for internal quality of MATLAB/Simulink/Stateflow models , 2012, 2012 IEEE International Conference on Industrial Technology.

[53]  Jean-Pierre Rosen,et al.  The SQALE Quality and Analysis Models for Assessing the Quality of Ada Source Code , 2011, Ada-Europe.

[54]  Joost Visser,et al.  A survey-based study of the mapping of system properties to ISO/IEC 9126 maintainability characteristics , 2009, 2009 IEEE International Conference on Software Maintenance.

[55]  Suhaimi Ibrahim,et al.  A taxonomy analysis for multi-model process improvement from the context of software engineering processes and services , 2012 .

[56]  Peng Shen,et al.  Research on Software Quality Assurance Based on Software Quality Standards and Technology Management , 2018, 2018 19th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD).

[57]  Reinhold Plösch,et al.  Objective Measurement of Safety in the Context of IEC 61508-3 , 2013, 2013 39th Euromicro Conference on Software Engineering and Advanced Applications.

[58]  Jean-Christophe Nebel,et al.  Quality assessment of protein model-structures based on structural and functional similarities , 2012, BMC Bioinformatics.

[59]  Stéphane Ducasse,et al.  Practices in the Squale Quality Model (Squale Deliverable 1.3) , 2010 .

[60]  Shari Lawrence Pfleeger,et al.  Software Quality: The Elusive Target , 1996, IEEE Softw..

[61]  Robert A. Martin,et al.  PROVIDING A FRAMEWORK FOR EFFECTIVE SOFTWARE QUALITY MEASUREMENT: MAKING A SCIENCE OF RISK ASSESSMENT , 1996 .

[62]  Jean-Louis Letouzey Managing Large Application Portfolio with Technical Debt Related Measures , 2016, 2016 Joint Conference of the International Workshop on Software Measurement and the International Conference on Software Process and Product Measurement (IWSM-MENSURA).

[63]  Barbara A. Kitchenham,et al.  The SQUID approach to defining a quality model , 1997, Software Quality Journal.

[64]  L. Benedicenti,et al.  A quality assessment model for Java code , 2002, IEEE CCECE2002. Canadian Conference on Electrical and Computer Engineering. Conference Proceedings (Cat. No.02CH37373).

[65]  Jürgen Münch,et al.  CQML Scheme: A Classification Scheme for Comprehensive Quality Model Landscapes , 2009, 2009 35th Euromicro Conference on Software Engineering and Advanced Applications.

[66]  Sepehr Forouzani,et al.  Method for Assessing Software Quality Using Source Code Analysis , 2016, ICNCC '16.

[67]  Marion Lepmets,et al.  A Quality Measurement Framework for IT Services , 2011, 2011 Annual SRII Global Conference.

[68]  R. Gulezian,et al.  Software quality measurement and modeling, maturity, control and improvement , 1995, Proceedings of Software Engineering Standards Symposium.

[69]  Taghi M. Khoshgoftaar,et al.  An assessment of software quality in a C++ environment , 1995, Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'95.

[70]  Sandra Sproge ASSESSMENT OF STUDY PROGRAMME QUALITY AT HIGHER EDUCATION INSTITUTION , 2012 .

[71]  Lars Nordström,et al.  Enterprise Architecture: A Framework Supporting System Quality Analysis , 2007, 11th IEEE International Enterprise Distributed Object Computing Conference (EDOC 2007).

[72]  Elli Georgiadou,et al.  GEQUAMO—A Generic, Multilayered, Customisable, Software Quality Model , 2003, Software Quality Journal.

[73]  Taghi M. Khoshgoftaar,et al.  Multivariate assessment of complex software systems: a comparative study , 1995, Proceedings of First IEEE International Conference on Engineering of Complex Computer Systems. ICECCS'95.

[74]  Alexandre Bergel,et al.  Assessing the Quality of your Software with MoQam , 2008 .

[75]  Kristina Pitula,et al.  On requirements elicitation for software projects in ict for development , 2010 .

[76]  Ali Idri,et al.  Applying ISO/IEC 25010 on Mobile Personal Health Records , 2015, HEALTHINF.

[77]  Frederick Webber,et al.  Static Software Metrics for Reliability and Maintainability , 2018, 2018 IEEE/ACM International Conference on Technical Debt (TechDebt).

[78]  Jonathan Streit,et al.  The Quamoco Quality Meta-Model , 2012 .

[79]  Souheil Khaddaj,et al.  A Proposed Adaptable Quality Model for Software Quality Assurance , 2005 .

[80]  Ali Idri,et al.  A Framework for Evaluating the Software Product Quality of Pregnancy Monitoring Mobile Personal Health Records , 2016, Journal of Medical Systems.

[81]  Peter Hegedus Revealing the Effect of Coding Practices on Software Maintainability , 2013, 2013 IEEE International Conference on Software Maintenance.

[82]  Juan Manuel Carrillo-de-Gea,et al.  An Empirical Study of Neural Network-Based Audience Response Technology in a Human Anatomy Course for Pharmacy Students , 2016, Journal of Medical Systems.

[83]  Abdelaziz Marzak,et al.  Towards a New Interoperability Quality Model for IoTs , 2018, 2018 Fifth International Symposium on Innovation in Information and Communication Technology (ISIICT).

[84]  Jiawen Lu,et al.  An Induction to the Development of Software Quality Model Standards , 2016, 2016 Third International Conference on Trustworthy Systems and their Applications (TSA).

[85]  Jean-Louis Letouzey,et al.  The SQALE method for evaluating Technical Debt , 2012, 2012 Third International Workshop on Managing Technical Debt (MTD).

[86]  Elmar Jürgens,et al.  Software quality models: Purposes, usage scenarios and requirements , 2009, 2009 ICSE Workshop on Software Quality.

[87]  Jürgen Münch,et al.  Adapting Software Quality Models: Practical Challenges, Approach, and First Empirical Results , 2011, 2011 37th EUROMICRO Conference on Software Engineering and Advanced Applications.

[88]  Raed Shatnawi,et al.  An Empirical Assessment of Refactoring Impact on Software Quality Using a Hierarchical Quality Model , 2011 .

[89]  Keun Lee,et al.  Software Quality Model for Consumer Electronics Product , 2009, 2009 Ninth International Conference on Quality Software.

[90]  Lotfi A. Zadeh,et al.  The concept of a linguistic variable and its application to approximate reasoning-III , 1975, Inf. Sci..

[91]  Ali Idri,et al.  Experiment design of free pregnancy monitoring mobile personal health records quality evaluation , 2016, 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom).

[92]  M Joseph,et al.  Software metrics: Establishing a company-wide program: by R B Grady and D L Caswell. Published by Prentice-Hall. 288pp. £37.80 , 1988 .

[93]  Mehrdad Sabetzadeh,et al.  Planning for Safety Standards Compliance: A Model-Based Tool-Supported Approach , 2012, IEEE Software.

[94]  Morgan Ericsson,et al.  A Study of the Effect of Data Normalization on Software and Information Quality Assessment , 2013, 2013 20th Asia-Pacific Software Engineering Conference (APSEC).

[95]  Simone Borges Simão Monteiro,et al.  Continuous Improvement of Systems in Maintenance Using a Proactive Quality Management , 2018, QUATIC.

[96]  Rodolfo Castello,et al.  An agile software quality framework lacking , 2014, 2014 World Congress on Computer Applications and Information Systems (WCCAIS).

[97]  Y. Raghu Reddy,et al.  Software Quality Models: A Systematic Mapping Study , 2019, 2019 IEEE/ACM International Conference on Software and System Processes (ICSSP).

[98]  Carl G. Davis,et al.  A Hierarchical Model for Object-Oriented Design Quality Assessment , 2002, IEEE Trans. Software Eng..

[99]  Francisco Chiclana,et al.  Modelling Execution Tracing Quality by Means of Type-1 Fuzzy Logic , 2013 .

[100]  Joost Visser,et al.  Benchmarking Technical Quality of Software Products , 2008, 2008 15th Working Conference on Reverse Engineering.

[101]  Suad Alramouni,et al.  A Quality Framework for Software Continuous Integration , 2015 .

[102]  Rüdiger Lincke,et al.  Comparing software metrics tools , 2008, ISSTA '08.

[103]  Yangyang Zhang,et al.  A Software Quality Quantifying Method Based on Preference and Benchmark Data , 2018, 2018 19th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD).

[104]  Markus Helfert,et al.  Information Quality and Accessibility , 2008, SCSS.

[105]  K. K. Aggarwal,et al.  Investigating effect of Design Metrics on Fault Proneness in Object-Oriented Systems , 2007, J. Object Technol..

[106]  Joost Visser,et al.  A Practical Model for Measuring Maintainability , 2007 .

[107]  Carsten Weise,et al.  Providing a Software Quality Framework for Testing of Mobile Applications , 2011, 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation.

[108]  Elli Georgiadou,et al.  In search for a widely applicable and accepted software quality model for software quality engineering , 2007, Software Quality Journal.

[109]  Aleksandar Milenkovic,et al.  Real-time unobtrusive program execution trace compression using branch predictor events , 2010, CASES '10.

[110]  Peter Hegedus A Probabilistic Quality Model for C# - an Industrial Case Study , 2013 .

[111]  Shing-Ko Liang,et al.  Selecting the Optimal ERP Software by Combining the ISO 9126 Standard and Fuzzy AHP Approach , 2006 .

[112]  Thierry Coq,et al.  The SQALE Analysis Model: An Analysis Model Compliant with the Representation Condition for Assessing the Quality of Software Source Code , 2010, 2010 Second International Conference on Advances in System Testing and Validation Lifecycle.

[113]  Barry W. Boehm,et al.  Quantitative evaluation of software quality , 1976, ICSE '76.

[114]  Miroslaw Staron,et al.  A Key Performance Indicator Quality Model and Its Industrial Evaluation , 2016, 2016 Joint Conference of the International Workshop on Software Measurement and the International Conference on Software Process and Product Measurement (IWSM-MENSURA).

[115]  Margaret Ross,et al.  Quality evaluation for Model-Driven Web Engineering methodologies , 2012, Inf. Softw. Technol..