Using Evolutionary Algorithms for Higher-Order Mutation Testing

Most software faults are complex higher-order mutants and their fixing needs more changes than first-order mutants. First-order mutants are created by inserting a single fault in the tested program. Higher-order mutants are created by injecting two or more faults in the tested program. Mutation testing has been developed to generate test inputs to kill the mutants of the tested program. Evolutionary algorithms have been effectively used in many software testing activities especially producing the required test inputs. In this paper, we introduce a genetic algorithm based technique to aid the automatic generation of test inputs for killing higher-order mutants. The proposed technique includes two policies: the first policy aims at killing the first-order mutants, and the second policy aims at killing the higher-order mutants. In addition, we introduce two new algorithms to generate the higher-order mutants. The paper also presents the results of the experiments that have been carried out to evaluate the effectiveness of our technique with its two policies. The results of the conducted empirical study showed that our proposed technique is more efficiency than random tests generation techniques in killing higher-order mutants.

[1]  Ahmed S. Ghiduk,et al.  Using Genetic Algorithms and Dominance Concepts for Generating Reduced Test Data , 2010, Informatica.

[2]  K. N. King,et al.  A fortran language system for mutation‐based software testing , 1991, Softw. Pract. Exp..

[3]  Lionel C. Briand,et al.  Is mutation an appropriate tool for testing experiments? , 2005, ICSE.

[4]  Zbigniew Michalewicz,et al.  Genetic algorithms + data structures = evolution programs (3rd ed.) , 1996 .

[5]  Mario Piattini,et al.  Decreasing the cost of mutation testing with second-order mutants , 2009 .

[6]  Mario Piattini,et al.  Decreasing the cost of mutation testing with second‐order mutants , 2009, Softw. Test. Verification Reliab..

[7]  Mark Harman,et al.  Multi Objective Higher Order Mutation Testing with Genetic Programming , 2009 .

[8]  Jonathan Timmis,et al.  Immune and Evolutionary Approaches to Software Mutation Testing , 2007, ICARIS.

[9]  Mark Harman,et al.  Higher Order Mutation Testing , 2009, Inf. Softw. Technol..

[10]  Ilene Burnstein,et al.  Practical Software Testing: A Process-Oriented Approach , 2003 .

[11]  Jinhui Shan,et al.  An Approach to Test Data Generation for Killing Multiple Mutants , 2006, 2006 22nd IEEE International Conference on Software Maintenance.

[12]  Shalini Kapoor,et al.  TEST CASE EFFECTIVENESS OF HIGHER ORDER MUTATION TESTING , 2011 .

[13]  A. Jefferson Offutt,et al.  Constraint-Based Automatic Test Data Generation , 1991, IEEE Trans. Software Eng..

[14]  Roy P. Pargas,et al.  Test-Data Generation Using Genetic Algorithms , 1999, Softw. Test. Verification Reliab..

[15]  Mark Harman,et al.  Constructing Subtle Faults Using Higher Order Mutation Testing , 2008, 2008 Eighth IEEE International Working Conference on Source Code Analysis and Manipulation.

[16]  Koushik Sen,et al.  DART: directed automated random testing , 2005, PLDI '05.

[17]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[18]  Mike Papadakis,et al.  Towards automating the generation of mutation tests , 2010, AST '10.

[19]  Mark Harman,et al.  An Analysis and Survey of the Development of Mutation Testing , 2011, IEEE Transactions on Software Engineering.

[20]  Peter S. May,et al.  Test data generation : two evolutionary approaches to mutation testing , 2007 .

[21]  A. Jefferson Offutt,et al.  MuJava: an automated class mutation system , 2005, Softw. Test. Verification Reliab..

[22]  Giuliano Antoniol,et al.  Automatic mutation test input data generation via ant colony , 2007, GECCO '07.

[23]  Mark Harman,et al.  Strong higher order mutation-based test data generation , 2011, ESEC/FSE '11.

[24]  Yu-Seung Ma.  Description of Method-level Mutation Operators for Java , 2011 .

[25]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[26]  Aderonke Olusola Akinde.  USING HIGHER ORDER MUTATION FOR REDUCING EQUIVALENT MUTANTS IN MUTATION TESTING , 2013 .

[27]  Mary Jean Harrold,et al.  Using Genetic Algorithms to Aid Test-Data Generation for Data-Flow Coverage , 2007, 14th Asia-Pacific Software Engineering Conference (APSEC'07).

[28]  Gary McGraw,et al.  Generating Software Test Data by Evolution , 2001, IEEE Trans. Software Eng..

[29]  William E. Howden,et al.  Weak Mutation Testing and Completeness of Test Sets , 1982, IEEE Transactions on Software Engineering.

[30]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[31]  Richard G. Hamlet,et al.  Testing Programs with the Aid of a Compiler , 1977, IEEE Transactions on Software Engineering.

[32]  Michael D. Ernst,et al.  Feedback-Directed Random Test Generation , 2007, 29th International Conference on Software Engineering (ICSE'07).

[33]  Bertrand Meyer,et al.  ARTOO , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[34]  Mark Harman,et al.  A Manifesto for Higher Order Mutation Testing , 2010, 2010 Third International Conference on Software Testing, Verification, and Validation Workshops.

[35]  A. Jefferson Offutt,et al.  An Experimental Comparison of Four Unit Test Criteria: Mutation, Edge-Pair, All-Uses and Prime Path Coverage , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[36]  Phyllis G. Frankl,et al.  All-uses vs mutation testing: An experimental comparison of effectiveness , 1997, J. Syst. Softw..

[37]  Ahmed S. Ghiduk.  Automatic Generation of Object-Oriented Tests with a Multistage-Based Genetic Algorithm , 2010, J. Comput..

[38]  Tsong Yueh Chen,et al.  Adaptive random testing through dynamic partitioning , 2004, Fourth International Conference onQuality Software, 2004. QSIC 2004. Proceedings..