Effective Regression Testing Using Requirements and Risks

The use of system requirements and their risks enables software testers to identify more important test cases that can reveal faults associated with risky components. Having identified those test cases, software testers can manage the testing schedule more effectively by running such test cases earlier so that they can fix faults sooner. Some work in this area has been done, but the previous approaches and studies have some limitations, such as an improper use of requirements risks in prioritization and an inadequate evaluation method. To address the limitations, we implemented a new requirements risk-based prioritization technique and evaluated it considering whether the proposed approach can detect faults earlier overall. It can also detect faults associated with risky components earlier. Our results indicate that the proposed approach is effective for detecting faults early and even better for finding faults associated with risky components of the system earlier than the existing techniques.

[1]  Gregg Rothermel,et al.  Modeling the cost-benefits tradeoffs for regression testing techniques , 2002, International Conference on Software Maintenance, 2002. Proceedings..

[2]  Laurie A. Williams,et al.  Prioritization of Regression Tests using Singular Value Decomposition with Empirical Change Records , 2007, The 18th IEEE International Symposium on Software Reliability (ISSRE '07).

[3]  Yanping Chen,et al.  Specification-based regression test selection with risk analysis , 2002, CASCON.

[4]  Hema Srikanth,et al.  On the Economics of Requirements-Based Test Case Prioritization , 2005 .

[5]  Amitabh Srivastava,et al.  Effectively prioritizing tests in development environment , 2002, ISSTA '02.

[6]  Mark Harman,et al.  Regression Testing Minimisation, Selection and Prioritisation - A Survey , 2009 .

[7]  Ståle Amland Risk-based testing: : Risk analysis fundamentals and metrics for software testing including a financial application case study , 2000, J. Syst. Softw..

[8]  Byoungju Choi,et al.  A Test Case Prioritization through Correlation of Requirement and Risk , 2012 .

[9]  S. A. Sahaaya Arul Mary,et al.  Factor oriented requirement coverage based system test case prioritization of new and regression test cases , 2009, Inf. Softw. Technol..

[10]  D. Richard Kuhn,et al.  FAILURE MODES IN MEDICAL DEVICE SOFTWARE: AN ANALYSIS OF 15 YEARS OF RECALL DATA , 2001 .

[11]  Gregg Rothermel,et al.  On the Use of Mutation Faults in Empirical Assessments of Test Case Prioritization Techniques , 2006, IEEE Transactions on Software Engineering.

[12]  Gregg Rothermel,et al.  Prioritizing test cases for regression testing , 2000, ISSTA '00.

[13]  Laurie A. Williams,et al.  System test case prioritization of new and regression test cases , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[14]  Hyunsook Do,et al.  Test Case Prioritization Using Requirements-Based Clustering , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[15]  Norman F. Schneidewind,et al.  An Experiment in Software Error Data Collection and Analysis , 1979, IEEE Transactions on Software Engineering.

[16]  Klaus Pohl,et al.  An automated technique for risk-based test case generation and prioritization , 2008, AST '08.

[17]  Gregg Rothermel,et al.  Test Case Prioritization: A Family of Empirical Studies , 2002, IEEE Trans. Software Eng..

[18]  Wayne M. Zage,et al.  Evaluating design metrics on large-scale software , 1993, IEEE Software.

[19]  Gregg Rothermel,et al.  The Effects of Time Constraints on Test Case Prioritization: A Series of Controlled Experiments , 2010, IEEE Transactions on Software Engineering.

[20]  Evangelos Triantaphyllou,et al.  The impact of aggregating benefit and cost criteria in four MCDA methods , 2005, IEEE Transactions on Engineering Management.

[21]  Myra B. Cohen,et al.  Configuration-aware regression testing: an empirical study of sampling and prioritization , 2008, ISSTA '08.

[22]  Harvey P. Siy,et al.  Predicting Fault Incidence Using Software Change History , 2000, IEEE Trans. Software Eng..

[23]  Mark Harman,et al.  Regression testing minimization, selection and prioritization: a survey , 2012, Softw. Test. Verification Reliab..

[24]  James Bach Risk and Requirements-Based Testing , 1999, Computer.