Genetic Programming for Multi-objective Test Data Generation in Search Based Software Testing

Software testing is an indispensable part in software development to ensure the quality of products. Multi-objective test data generation is a sub-area of search-based software testing, which focuses on automatically generating test data to form high quality test suites. Due to the limited data representation and the lack of specific multi-objective optimization methods, existing approaches have drawbacks in dealing with real-world programs. This paper presents a new approach to multi-objective test data generation problems using genetic programming (GP), while two genetic algorithm (GA) based approaches are also implemented for comparison purposes. Furthermore, three multi-objective optimization frameworks are used and compared to examine the performance of the GP-based methods. Experiments have been conducted on two types of test data generation problems: integer and double. Each consists of 160 benchmark programs with different degrees of nesting. The results suggest that the new GP approaches perform much better than the two GA-based approaches, and a random search baseline algorithm.

[1]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[2]  Bernhard K. Aichernig,et al.  Survey on test data generation tools , 2013, International Journal on Software Tools for Technology Transfer.

[3]  Mark Harman,et al.  A multi-objective approach to search-based test data generation , 2007, GECCO '07.

[4]  Bahriye Akay,et al.  Comparisons of metaheuristic algorithms and fitness functions on software test data generation , 2016, Appl. Soft Comput..

[5]  Yuanyuan Zhang,et al.  Achievements, Open Problems and Challenges for Search Based Software Testing , 2015, 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST).

[6]  Wasif Afzal,et al.  A systematic review of search-based testing for non-functional system properties , 2009, Inf. Softw. Technol..

[7]  Silvia Regina Vergilio,et al.  A Multi-Objective Genetic Algorithm to Test Data Generation , 2010, 2010 22nd IEEE International Conference on Tools with Artificial Intelligence.

[8]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[9]  Francisco Luna,et al.  MOCell: A cellular genetic algorithm for multiobjective optimization , 2009 .

[10]  Xin Yao,et al.  Multi-Objective Approaches to Optimal Testing Resource Allocation in Modular Software Systems , 2010, IEEE Transactions on Reliability.

[11]  Enrique Alba,et al.  Benchmark Generator for Software Testers , 2011, EANN/AIAI.

[12]  Enrique Alba,et al.  Evolutionary algorithms for the multi‐objective test data generation problem , 2012, Softw. Pract. Exp..

[13]  Mary Jean Harrold,et al.  Using Genetic Algorithms to Aid Test-Data Generation for Data-Flow Coverage , 2007, 14th Asia-Pacific Software Engineering Conference (APSEC'07).

[14]  Lars Grunske,et al.  Test data generation with a Kalman filter-based adaptive genetic algorithm , 2015, J. Syst. Softw..

[15]  Mark Harman,et al.  Strong higher order mutation-based test data generation , 2011, ESEC/FSE '11.

[16]  John R. Koza,et al.  Introduction to genetic programming tutorial: from the basics to human-competitive results , 2010, GECCO '10.

[17]  Norbert Oster,et al.  Automatic Test Data Generation by Multi-objective Optimisation , 2006, SAFECOMP.