A Systematic Requirements and Risks-Based Test Case Prioritization Using a Fuzzy Expert System

The use of risk information can help software engineers identify software components that are likely vulnerable or require extra attention when testing. Some studies have shown that the requirements risk-based approaches can be effective in improving the effectiveness of regression testing techniques. However, the risk estimation processes used in such approaches can be subjective, time-consuming, and costly. In this research, we introduce a fuzzy expert system that emulates human thinking to address the subjectivity related issues in the risk estimation process in a systematic and an efficient way and thus further improve the effectiveness of test case prioritization. Further, the required data for our approach was gathered by employing a semi-automated process that made the risk estimation process less subjective. The empirical results indicate that the new prioritization approach can improve the rate of fault detection over several existing test case prioritization techniques, while reducing threats to subjective risk estimation.

[1]  Byoungju Choi,et al.  Effective Regression Testing Using Requirements and Risks , 2014, 2014 Eighth International Conference on Software Security and Reliability.

[2]  Ina Schieferdecker,et al.  A taxonomy of risk-based testing , 2014, International Journal on Software Tools for Technology Transfer.

[3]  Byoungju Choi,et al.  Risk-based test case prioritization using a fuzzy expert system , 2016, Inf. Softw. Technol..

[4]  Byoungju Choi,et al.  A Test Case Prioritization through Correlation of Requirement and Risk , 2012 .

[5]  Taghi M. Khoshgoftaar,et al.  System regression test planning with a fuzzy expert system , 2014, Inf. Sci..

[6]  Moataz A. Ahmed,et al.  Adaptive fuzzy logic-based framework for software development effort prediction , 2005, Inf. Softw. Technol..

[7]  Michael Hadjimichael,et al.  A fuzzy expert system for aviation risk assessment , 2009, Expert Syst. Appl..

[8]  Amitabh Srivastava,et al.  Effectively prioritizing tests in development environment , 2002, ISSTA '02.

[9]  Wayne M. Zage,et al.  Evaluating design metrics on large-scale software , 1993, IEEE Software.

[10]  Harleen Kaur,et al.  Design and Implementation of Fuzzy Expert System for Back pain Diagnosis , 2011 .

[11]  M. Neshat,et al.  A Fuzzy Expert System for Heart Disease Diagnosis , 2022 .

[12]  Servet Soyguder,et al.  Predicting of fan speed for energy saving in HVAC system based on adaptive network based fuzzy inference system , 2009, Expert Syst. Appl..

[13]  Taghi M. Khoshgoftaar,et al.  Application of fuzzy expert system in test case selection for system regression test , 2005, IRI -2005 IEEE International Conference on Information Reuse and Integration, Conf, 2005..

[14]  Gregg Rothermel,et al.  The Effects of Time Constraints on Test Case Prioritization: A Series of Controlled Experiments , 2010, IEEE Transactions on Software Engineering.

[15]  Joseph H. M. Tah,et al.  A fuzzy approach to constuction project risk assessment and analysis: construction project risk management system , 2001 .

[16]  Hyunsook Do,et al.  Requirements based test prioritization using risk factors: An industrial study , 2016, Inf. Softw. Technol..

[17]  Gregg Rothermel,et al.  Prioritizing test cases for regression testing , 2000, ISSTA '00.

[18]  Yan Li,et al.  Approaches for the combined use of risk analysis and testing: a systematic literature review , 2014, International Journal on Software Tools for Technology Transfer.

[19]  T. Saaty,et al.  The Analytic Hierarchy Process , 1985 .

[20]  Ståle Amland Hulda Garborgsv.  Risk Based Testing and Metrics Risk Analysis Fundamentals and Metrics for software testing including a Financial Application case study , 2002 .

[21]  Andreas Metzger,et al.  Employing Requirements Metrics for Automating Early Risk Assessment , 2007 .

[22]  M. Neshat,et al.  Fuzzy Expert System Design for Diagnosis of Liver Disorders , 2008, 2008 International Symposium on Knowledge Acquisition and Modeling.

[23]  Klaus Pohl,et al.  An automated technique for risk-based test case generation and prioritization , 2008, AST '08.

[24]  Gregg Rothermel,et al.  Test Case Prioritization: A Family of Empirical Studies , 2002, IEEE Trans. Software Eng..

[25]  Ståle Amland.  Risk-based testing: : Risk analysis fundamentals and metrics for software testing including a financial application case study , 2000, J. Syst. Softw..

[26]  E. H. Mamdani,et al.  An Experiment in Linguistic Synthesis with a Fuzzy Logic Controller , 1999, Int. J. Man Mach. Stud..

[27]  S. A. Sahaaya Arul Mary,et al.  Factor oriented requirement coverage based system test case prioritization of new and regression test cases , 2009, Inf. Softw. Technol..

[28]  Barry W. Boehm,et al.  Improving software testing process: feature prioritization to make winners of success‐critical stakeholders , 2012, J. Softw. Evol. Process..

[29]  Hyunsook Do,et al.  A Fuzzy Expert System for Cost-Effective Regression Testing Strategies , 2013, 2013 IEEE International Conference on Software Maintenance.

[30]  Norman F. Schneidewind,et al.  An Experiment in Software Error Data Collection and Analysis , 1979, IEEE Transactions on Software Engineering.

[31]  Laurie A. Williams,et al.  System test case prioritization of new and regression test cases , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[32]  Yashwant K. Malaiya,et al.  Requirements volatility and defect density , 1999, Proceedings 10th International Symposium on Software Reliability Engineering (Cat. No.PR00443).

[33]  Hyunsook Do,et al.  Test Case Prioritization Using Requirements-Based Clustering , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[34]  Laurie A. Williams,et al.  Towards the prioritization of system test cases , 2014, Softw. Test. Verification Reliab..

[35]  Nasser Ghasem-Aghaee,et al.  Fuzzy Emotional COCOMO II Software Cost Estimation (FECSCE) using Multi-Agent Systems , 2011, Appl. Soft Comput..

[36]  Mark Harman,et al.  Regression testing minimization, selection and prioritization: a survey , 2012, Softw. Test. Verification Reliab..

[37]  James Bach.  Risk and Requirements-Based Testing , 1999, Computer.

[38]  Ruth Breu,et al.  A Risk Assessment Framework for Software Testing , 2014, ISoLA.

[39]  D. Richard Kuhn,et al.  FAILURE MODES IN MEDICAL DEVICE SOFTWARE: AN ANALYSIS OF 15 YEARS OF RECALL DATA , 2001 .

[40]  Thomas L. Saaty,et al.  DECISION MAKING WITH THE ANALYTIC HIERARCHY PROCESS , 2008 .

[41]  Yanping Chen,et al.  Specification-based regression test selection with risk analysis , 2002, CASCON.

[42]  Gregg Rothermel,et al.  Modeling the cost-benefits tradeoffs for regression testing techniques , 2002, International Conference on Software Maintenance, 2002. Proceedings..

[43]  Laurie A. Williams,et al.  Prioritization of Regression Tests using Singular Value Decomposition with Empirical Change Records , 2007, The 18th IEEE International Symposium on Software Reliability (ISSRE '07).

[44]  Eric W. T. Ngai,et al.  Fuzzy decision support system for risk analysis in e-commerce development , 2005, Decis. Support Syst..