Analyzing and documenting the systematic review results of software testing ontologies

Abstract Context Software testing is a complex area since it has a large number of specific methods, processes and strategies, involving a lot of domain concepts. Therefore, it would be valuable to have a conceptualized software testing ontology that explicitly and unambiguously defines the concepts. Consequently, it is important to find out the available evidence in the literature on primary studies for software testing ontologies. In particular, we are looking for research that has a rich ontological coverage that includes Non-Functional Requirements (NFRs) and Functional Requirements (FRs) concepts in conjunction with static and dynamic testing concepts, which can be used in method and process specifications for a family of testing strategies. Objective The main goal for this secondary study is to identify, evaluate and synthesize the available primary studies on conceptualized software testing ontologies. Method To conduct this study, we use the Systematic Literature Review (SLR) approach, which follows our enhanced SLR process. We set three research questions. Additionally, to quantitatively evaluate the quality of the selected conceptualized ontologies, we designed a NFRs tree and its associated metrics and indicators. Results We obtained 12 primary studies documenting conceptualized testing ontologies by using three different retrieval methods. In general, we noted that most of them have a lack of NFRs and static testing terminological coverage. Finally, we observe that none of them is directly linked with FRs and NFRs conceptual components. Conclusion A general benefit of having the suitable software testing ontology is to minimize the current heterogeneity, ambiguity and incompleteness problems in terms, properties and relationships. We have confirmed that exists heterogeneity, ambiguity, and incompleteness for concepts dealing with testing artifacts, roles, activities, and methods. Moreover, we did not find the suitable ontology for our aim since none of the conceptualized ontologies are directly linked with NFRs and FRs components.

[1]  Zhenyu Liu,et al.  Test Case Reuse Based on Ontology , 2009, 2009 15th IEEE Pacific Rim International Symposium on Dependable Computing.

[2]  Pablo Becker,et al.  A Systematic Review on Software Testing Ontologies , 2019, QUATIC.

[3]  Asunción Gómez-Pérez,et al.  ONTOMETRIC: A Method to Choose the Appropriate Ontology , 2004, J. Database Manag..

[4]  Pearl Brereton,et al.  Lessons from applying the systematic literature review process within the software engineering domain , 2007, J. Syst. Softw..

[5]  Pearl Brereton,et al.  Evidence-Based Software Engineering and Systematic Reviews , 2015 .

[6]  Wei-Tek Tsai,et al.  Ontology-Based Test Modeling and Partition Testing of Web Services , 2008, 2008 IEEE International Conference on Web Services.

[7]  Pablo Becker,et al.  Linking Business and Information Need Goals with Functional and Non-functional Requirements , 2018, CIbSE.

[8]  Elisa Yumi Nakagawa,et al.  Ontology-based Development of Testing Related Tools , 2008, SEKE.

[9]  Aldo Gangemi,et al.  Modelling Ontology Evaluation and Validation , 2006, ESWC.

[10]  Vahid Garousi,et al.  A systematic literature review of literature reviews in software testing , 2016, Inf. Softw. Technol..

[11]  Hong Zhu,et al.  Developing Software Testing Ontology in UML for a Software Growth Environment of Web-Based Applications , 2005 .

[12]  Jing Tian,et al.  An Ontology-Based Knowledge Sharing Portal for Software Testing , 2017, 2017 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C).

[13]  Nandamudi Lankalapalli Vijaykumar,et al.  Practical similarities and differences between Systematic Literature Reviews and Systematic Mappings: a tertiary study , 2017, SEKE.

[14]  Raymond W Lam,et al.  Using Metaanalysis to Evaluate Evidence: Practical Tips and Traps , 2005, Canadian journal of psychiatry. Revue canadienne de psychiatrie.

[15]  Ricardo de Almeida Falbo,et al.  Ontologies in Software Testing: A Systematic Literature Review , 2013, ONTOBRAS.

[16]  Marko Grobelnik,et al.  A SURVEY OF ONTOLOGY EVALUATION TECHNIQUES , 2005 .

[17]  Barbara Kitchenham,et al.  Procedures for Performing Systematic Reviews , 2004 .

[18]  Nathalie Aussenac-Gilles,et al.  OQuaRE: A SQuaRE-based Approach for Evaluating the Quality of Ontologies , 2011, J. Res. Pract. Inf. Technol..

[19]  Pablo Becker,et al.  Family of Strategies for Different Evaluation Purposes , 2017, CIbSE.

[20]  Pablo Becker,et al.  An integrated strategy to systematically understand and manage quality in use for web applications , 2011, Requirements Engineering.

[21]  Carolyn L. Talcott,et al.  Document Logic: Risk analysis of business processes through document authenticity , 2009, 2009 13th Enterprise Distributed Object Computing Conference Workshops.

[22]  Auriol Degbelo,et al.  A Snapshot of Ontology Evaluation Criteria and Strategies , 2017, SEMANTiCS.

[23]  Pablo Becker,et al.  Applying an Improving Strategy that embeds Functional and Non-Functional Requirements Concepts , 2019, J. Comput. Sci. Technol..

[24]  Luis Olsina,et al.  How to Measure and Evaluate Web Applications in a Consistent Way , 2008, Web Engineering.

[25]  Letha H. Etzkorn,et al.  Cohesion Metrics for Ontology Design and Application , 2005 .

[26]  Michael Grüninger,et al.  Applied Ontology: A foreword by the new Editors-in-Chief , 2017, Appl. Ontology.

[27]  Jianwen Xiang,et al.  An Ontology-Based Knowledge Framework for Software Testing , 2017 .

[28]  Luis Olsina,et al.  Chapter 2 – Metrics and Indicators as Key Organizational Assets for ICT Security Assessment , 2014 .

[29]  Armin Dr.-Ing. habil. Zimmermann Software and Systems Engineering , 2010 .

[30]  Pablo Becker,et al.  Family of Evaluation Strategies: A Practical Case for Comparing and Adopting Strengths , 2018 .

[31]  Pablo Becker,et al.  Process Ontology Specification for Enhancing the Process Compliance of a Measurement and Evaluation Strategy , 2015, CLEI Electron. J..

[32]  Gustavo Rossi,et al.  Assessing the quality of academic websites: A case study , 1999, New Rev. Hypermedia Multim..

[33]  Gustavo Rossi,et al.  Measuring Web Application Quality with WebQEM , 2002, IEEE Multim..

[34]  Renata Vieira,et al.  An Ontology for Guiding Performance Testing , 2014, 2014 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT).

[35]  Kai Petersen,et al.  Guidelines for conducting systematic mapping studies in software engineering: An update , 2015, Inf. Softw. Technol..

[36]  Pablo Becker,et al.  Specifying the Process Model for Systematic Reviews: An Augmented Proposal , 2019, J. Softw. Eng. Res. Dev..

[37]  Guntis Arnicans,et al.  Semi-Automatic Generation of a Software Testing Lightweight Ontology from a Glossary Based on the ONTO6 Methodology , 2012, DB&IS.