Factors affecting the use of genetic algorithms in test suite augmentation

Test suite augmentation techniques are used in regression testing to help engineers identify code elements affected by changes, and generate test cases to cover those elements. Researchers have created various approaches to identify affected code elements, but only recently have they considered integrating, with this task, approaches for generating test cases. In this paper we explore the use of genetic algorithms in test suite augmentation. We identify several factors that impact the effectiveness of this approach, and we present the results of a case study exploring the effects of one of these factors: the manner in which existing and newly generated test cases are utilized by the genetic algorithm. Our results reveal several ways in which this factor can influence augmentation results, and reveal open problems that researchers must address if they wish to create augmentation techniques that make use of genetic algorithms.

[1]  Alessandro Orso,et al.  Test-Suite Augmentation for Evolving Software , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[2]  Andrea Arcuri,et al.  It Does Matter How You Normalise the Branch Distance in Search Based Software Testing , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[3]  Bryan F. Jones,et al.  Automatic structural testing using genetic algorithms , 1996, Softw. Eng. J..

[4]  Hélène Waeselynck,et al.  Simulated annealing applied to test generation: landscape characterization and stopping criteria , 2007, Empirical Software Engineering.

[5]  Paolo Tonella,et al.  Evolutionary testing of classes , 2004, ISSTA '04.

[6]  Joachim Wegener,et al.  Evolutionary test environment for automatic structural testing , 2001, Inf. Softw. Technol..

[7]  Roy P. Pargas,et al.  Test‐data generation using genetic algorithms , 1999, Softw. Test. Verification Reliab..

[8]  Robert S. Arnold,et al.  Software Change Impact Analysis , 1996 .

[9]  Phil McMinn,et al.  Search-based software test data generation: a survey: Research Articles , 2004 .

[10]  Phil McMinn,et al.  Search‐based software test data generation: a survey , 2004, Softw. Test. Verification Reliab..

[11]  David W. Binkley,et al.  Semantics Guided Regression Test Cost Reduction , 1997, IEEE Trans. Software Eng..

[12]  Alessandro Orso,et al.  MATRIX: Maintenance-Oriented Testing Requirements Identifier and Examiner , 2006, Testing: Academic & Industrial Conference - Practice And Research Techniques (TAIC PART'06).

[13]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[14]  Mary Lou Soffa,et al.  Program Slicing‐Based Regression Testing Techniques , 1996 .

[15]  Mark Harman,et al.  Test data augmentation : generating new test data from existing test data Technical Report : TR-0804 , 2008 .

[16]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[17]  Matthew B. Dwyer,et al.  Differential symbolic execution , 2008, SIGSOFT '08/FSE-16.

[18]  José Javier Dolado,et al.  A tabu search algorithm for structural software testing , 2008, Comput. Oper. Res..

[19]  Mark Harman,et al.  Search Algorithms for Regression Test Case Prioritization , 2007, IEEE Transactions on Software Engineering.

[20]  Stefan Wappler,et al.  Using evolutionary algorithms for the unit testing of object-oriented software , 2005, GECCO '05.

[21]  Zhihong Xu,et al.  Directed test suite augmentation , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[22]  Gregg Rothermel,et al.  Sofya: A Flexible Framework for Development of Dynamic Program Analyses for Java Software , 2006 .

[23]  Mark Harman,et al.  Test data regeneration: generating new test data from existing test data , 2012, Softw. Test. Verification Reliab..

[24]  Gregg Rothermel,et al.  Selecting tests and identifying test coverage requirements for modified software , 1994, ISSTA '94.

[25]  Gregg Rothermel,et al.  A safe, efficient regression test selection technique , 1997, TSEM.