Test suite generation with memetic algorithms

Genetic Algorithms have been successfully applied to the generation of unit tests for classes, and are well suited to create complex objects through sequences of method calls. However, because the neighborhood in the search space for method sequences is huge, even supposedly simple optimizations on primitive variables (e.g., numbers and strings) can be ineffective or unsuccessful. To overcome this problem, we extend the global search applied in the EVOSUITE test generation tool with local search on the individual statements of method sequences. In contrast to previous work on local search, we also consider complex datatypes including strings and arrays. A rigorous experimental methodology has been applied to properly evaluate these new local search operators. In our experiments on a set of open source classes of different kinds (e.g., numerical applications and text processing), the resulting test data generation technique increased branch coverage by up to 32% on average over the normal Genetic Algorithm.

[1]  Mohammad Alshraideh,et al.  Search‐based software test data generation for string data using program‐specific search operators , 2006, Softw. Test. Verification Reliab..

[2]  Gordon Fraser,et al.  On Parameter Tuning in Search Based Software Engineering , 2011, SSBSE.

[3]  Phil McMinn,et al.  Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[4]  Gordon Fraser,et al.  Whole Test Suite Generation , 2013, IEEE Transactions on Software Engineering.

[5]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[6]  Gordon Fraser,et al.  The Seed is Strong: Seeding Strategies in Search-Based Software Testing , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[7]  Gordon Fraser,et al.  Bytecode Testability Transformation , 2011, SSBSE.

[8]  Lionel C. Briand,et al.  A Hitchhiker's guide to statistical tests for assessing randomized algorithms in software engineering , 2014, Softw. Test. Verification Reliab..

[9]  Xin Yao,et al.  A Memetic Algorithm for test data generation of Object-Oriented software , 2007, 2007 IEEE Congress on Evolutionary Computation.

[10]  McMinnPhil Search-based software test data generation: a survey , 2004 .

[11]  Andrea Arcuri,et al.  Theoretical Analysis of Local Search in Software Testing , 2009, SAGA.

[12]  AlshraidehMohammad,et al.  Search-based software test data generation for string data using program-specific search operators , 2006 .

[13]  Pablo Moscato,et al.  On Evolution, Search, Optimization, Genetic Algorithms and Martial Arts : Towards Memetic Algorithms , 1989 .

[14]  Andreas Zeller,et al.  Mutation-Driven Generation of Unit Tests and Oracles , 2010, IEEE Transactions on Software Engineering.

[15]  Mark Harman,et al.  A theoretical & empirical analysis of evolutionary testing and hill climbing for structural test data generation , 2007, ISSTA '07.

[16]  Mark Harman,et al.  Testability transformation , 2004, IEEE Transactions on Software Engineering.

[17]  Mark Harman,et al.  A Theoretical and Empirical Study of Search-Based Testing: Local, Global, and Hybrid Search , 2010, IEEE Transactions on Software Engineering.

[18]  Gordon Fraser,et al.  Sound empirical evidence in software testing , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[19]  Bogdan Korel,et al.  Automated Software Test Data Generation , 1990, IEEE Trans. Software Eng..

[20]  Paolo Tonella,et al.  Evolutionary testing of classes , 2004, ISSTA '04.

[21]  Bogdan Korel,et al.  The chaining approach for software test data generation , 1996, TSEM.

[22]  Luciano Baresi,et al.  TestFul: An Evolutionary Test Approach for Java , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.