Reduction-based methods and metrics for selective regression testing

In corrective maintenance, modified software is regression tested using selected test cases in order to ensure that the modifications have not caused adverse effects. This activity of selective regression testing involves regression test selection, which refers to selecting test cases from the previously run test suite, and test-coverage identification. In this paper, we propose three test-selection methods and two coverage identification metrics. The three methods aim to reduce the number of selected test cases for retesting the modified software. The first method, referred to as modification-based reduction version 1 (MBR1), selects a reduced number of test cases based on the modification made and its effects in the software. The second method, referred to as modification-based reduction version 2 (MBR2) improves MBR1 by further omitting tests that do not cover the modification. The third method, referred to as precise reduction (PR), reduces the number of test cases selected by omitting non-modification-revealing tests from the initial test suite. The two coverage metrics are McCabe-based regression test metrics, which are referred to as the Reachability regression Test selection McCabe-based metric (RTM), and data-flow Slices regression Test McCabe-based metric (STM). These metrics aim to assist the regression tester in monitoring test-coverage adequacy, reveal any shortage or redundancy in the test suite, and assist in identifying, where additional tests may be required for regression testing. We empirically compare MBR1, MBR2, and PR with three reduction and precision-oriented methods on 60 test-problems. The results show that PR selects the least number of test cases most of the time and omits non-modification-revealing tests. We also demonstrate the applicability of our proposed methods to object-oriented regression testing at the class level. Further, we illustrate typical application of the RTM and STM metrics using the 60 test-problems and two coverage-oriented selective regression-testing methods.

[1]  Phyllis G. Frankl,et al.  Pythia: a regression test selection tool based on textual differencing , 1997 .

[2]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[3]  Gregg Rothermel,et al.  A safe, efficient regression test selection technique , 1997, TSEM.

[4]  Mary Lou Soffa,et al.  A methodology for controlling the size of a test suite , 1993, TSEM.

[5]  Gregg Rothermel,et al.  An empirical study of regression test selection techniques , 1998, Proceedings of the 20th International Conference on Software Engineering.

[6]  Nashat Mansour,et al.  Simulated Annealing and Genetic Algorithms for Optimal Regression Testing , 1999, J. Softw. Maintenance Res. Pract..

[7]  Mark Priestley.  Practical Object-Oriented Design , 1996 .

[8]  David Chenho Kung,et al.  A technique for the selective revalidation of OO software , 1997, J. Softw. Maintenance Res. Pract..

[9]  Hareton Leung,et al.  A firewall concept for both control-flow and data-flow in regression integration testing , 1992, Proceedings Conference on Software Maintenance 1992.

[10]  Rami Bahsoon,et al.  Empirical comparison of regression test selection algorithms , 2001, J. Syst. Softw..

[11]  Rajiv Gupta,et al.  An approach to regression testing using slicing , 1992, Proceedings Conference on Software Maintenance 1992.

[12]  David W. Binkley,et al.  Semantics Guided Regression Test Cost Reduction , 1997, IEEE Trans. Software Eng..

[13]  Gregg Rothermel,et al.  Analyzing Regression Test Selection Techniques , 1996, IEEE Trans. Software Eng..

[14]  Ning Zhang,et al.  Automated regression testing using DBT and Sleuth , 1999 .

[15]  Mary Lou Soffa,et al.  Program Slicing‐Based Regression Testing Techniques , 1996 .

[16]  David Chenho Kung,et al.  Or Regression Testing of Object-Oriented Programs , 1996, J. Syst. Softw..

[17]  Joseph Robert Horgan,et al.  Incremental regression testing , 1993, 1993 Conference on Software Maintenance.