Search-Based Propagation of Regression Faults in Automated Regression Testing

Over the lifetime of software programs, developers make changes by adding, removing, enhancing functionality or by refactoring code. These changes can sometimes result in undesired side effects in the original functionality of the software, better known as regression faults. To detect these, developers either have to rely on an existing set of test cases, or have to create new tests that exercise the changes. However, simply executing the changed code does not guarantee that a regression fault manifests in a state change, or that this state change propagates to an observable output where it could be detected by a test case. To address this propagation aspect, we present EVOSUITER, an extension of the EVOSUITE unit test generation tool. Our approach generates tests that propagate regression faults to an observable difference using a search-based approach, and captures this observable difference with test assertions. We illustrate on an example program that EVOSUITER can be effective in revealing regression errors in cases where alternative approaches may fail, and motivate further research in this direction.

[1]  Gordon Fraser,et al.  Evolutionary Generation of Whole Test Suites , 2011, 2011 11th International Conference on Quality Software.

[2]  Raúl A. Santelices,et al.  Applying aggressive propagation-based strategies for testing changes , 2011, 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation.

[3]  Alessandro Orso,et al.  JDiff: A differencing technique and tool for object-oriented programs , 2006, Automated Software Engineering.

[4]  Tao Xie,et al.  DiffGen: Automated Regression Unit-Test Generation , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[5]  Michael D. Ernst,et al.  Randoop: feedback-directed random testing for Java , 2007, OOPSLA '07.

[6]  McMinnPhil Search-based software test data generation: a survey , 2004 .

[7]  Rajiv Gupta,et al.  A methodology for controlling the size of a test suite , 1990, Proceedings. Conference on Software Maintenance 1990.

[8]  Zhihong Xu,et al.  Directed test suite augmentation , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[9]  Alessandro Orso,et al.  Automated Behavioral Regression Testing , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[10]  Alessandro Orso,et al.  BERT: BEhavioral Regression Testing , 2008, WODA '08.

[11]  Jeffrey M. Voas,et al.  PIE: A Dynamic Failure-Based Technique , 1992, IEEE Trans. Software Eng..

[12]  Alessandro Orso,et al.  Test-Suite Augmentation for Evolving Software , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[13]  Bertrand Meyer,et al.  Object distance and its application to adaptive random testing of object-oriented programs , 2006, RT '06.

[14]  Gregg Rothermel,et al.  Prioritizing test cases for regression testing , 2000, ISSTA '00.

[15]  Nikolai Tillmann,et al.  eXpress: guided path exploration for efficient regression test generation , 2011, ISSTA '11.