Code coverage optimisation in genetic algorithms and particle swarm optimisation for automatic software test data generation

Automatic software test data generation is the process of generating a set of test cases for a given program which can achieve a high code coverage. Genetic algorithms (GAs) and particle swarm optimisation (PSO) can automatically evolve a set of test data, but the traditional representation in GAs and PSO produces solutions with a single set of data cases, which may not achieve good performance on programs with many complex conditions. This paper proposes a multi-vector representation in GAs and PSO, which can generate multiple sets of data cases in a single run, to generate test data for complex test programs. Experiments have been conducted to examine and compare the performance of GAs and PSO on six commonly used benchmark test programs and three newly developed programs with a relatively large number of complex conditions. The experimental results show that the proposed multi-vector representation can improve the performance of GAs and PSO on all the nine tested programs, achieving the optimal 100% code coverage on the relatively easy programs. PSO outperforms GAs in terms of both the code coverage and the computational efficiency, especially on the hard programs.

[1]  Mengjie Zhang,et al.  Particle Swarm Optimization for Feature Selection in Classification: A Multi-Objective Approach , 2013, IEEE Transactions on Cybernetics.

[2]  Xin Yao,et al.  Search based software testing of object-oriented containers , 2008, Inf. Sci..

[3]  Hans-Gerhard Groß,et al.  A Genetic Programming Approach to Automated Test Generation for Object-Oriented Software , 2006, Int. Trans. Syst. Sci. Appl..

[4]  Joachim Wegener,et al.  Applying particle swarm optimization to software testing , 2007, GECCO '07.

[5]  Andries P. Engelbrecht,et al.  Computational Intelligence: An Introduction , 2002 .

[6]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[7]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[8]  Audris Mockus,et al.  Test coverage and post-verification defects: A multiple case study , 2009, ESEM 2009.

[9]  Phil McMinn,et al.  Search-Based Software Testing: Past, Present and Future , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[10]  Mengjie Zhang,et al.  A Dimension Reduction Approach to Classification Based on Particle Swarm Optimisation and Rough Set Theory , 2012, Australasian Conference on Artificial Intelligence.

[11]  Roy P. Pargas,et al.  Test‐data generation using genetic algorithms , 1999, Softw. Test. Verification Reliab..

[12]  Phil McMinn,et al.  Search‐based software test data generation: a survey , 2004, Softw. Test. Verification Reliab..

[13]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[14]  Harmen-Hinrich Sthamer,et al.  The automatic generation of software test data using genetic algorithms , 1995 .

[15]  Timo Mantere,et al.  Genetic Algorithm Based Software Testing , 1997, ICANNGA.

[16]  Donald J. Berndt,et al.  Breeding software test cases with genetic algorithms , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[17]  Generating Test Data for Structural Testing Based on Ant Colony Optimization , 2012, 2012 12th International Conference on Quality Software.

[18]  Jun Zhang,et al.  Generating Software Test Data by Particle Swarm Optimization , 2014, SEAL.

[19]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[20]  Gary McGraw,et al.  Genetic algorithms for dynamic test data generation , 1997, Proceedings 12th IEEE International Conference Automated Software Engineering.

[21]  Mengjie Zhang,et al.  Novel Initialisation and Updating Mechanisms in PSO for Feature Selection in Classification , 2013, EvoApplications.

[22]  Mark Harman,et al.  Ieee Transactions on Evolutionary Computation 1 , 2022 .

[23]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[24]  Emil M. Petriu,et al.  Dynamic white-box software testing using a recursive hybrid evolutionary strategy/genetic algorithm , 2013, 2013 IEEE Congress on Evolutionary Computation.

[25]  Gary McGraw,et al.  Generating Software Test Data by Evolution , 2001, IEEE Trans. Software Eng..

[26]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[27]  A. Jefferson Offutt,et al.  A semantic model of program faults , 1996, ISSTA '96.

[28]  Hitesh Tahbildar,et al.  Heuristic Approach of Automated Test Data Generation for Program having Array of Different Dimensions and Loops with Variable Number of Iteration , 2010, ArXiv.

[29]  Nikolai Tillmann,et al.  Guided test generation for coverage criteria , 2010, 2010 IEEE International Conference on Software Maintenance.

[30]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..