Reduction in Mutation Testing of Java classes

In mutation analysis many simple modification of the original program called “mutants” are created. Test cases which are supposed to identify the introduced program changes are designed. Each mutant must be “killed” by a test case, i.e. the test case should detect the purposely introduced modification. Mutation testing is known to be effective but computationally demanding and time consuming because a large number of mutants has to be tested. Mutation score, which is the fraction of mutants that are killed by a test set, is often used to evaluate the effectiveness of mutation testing. An interesting research question is if the number of mutants can be reduced without significantly decreasing the effectiveness of the test. We were exploring selective reductions of mutants generated for Java programs. The results of several experiments conducted in the Eclipse environment are presented in this paper. These results show that selective reduction in mutants can significantly reduce the cost of testing with acceptable mutation score and code coverage.

[1]  A. Jefferson Offutt,et al.  Designing Deletion Mutation Operators , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation.

[2]  A. Nadeem,et al.  Object oriented mutation testing: A survey , 2012, 2012 International Conference on Emerging Technologies.

[3]  Macario Polo,et al.  Parallel mutation testing , 2013, Softw. Test. Verification Reliab..

[4]  Ilona Bluemke,et al.  A Comparison of Dataflow and Mutation Testing of Java Methods , 2011 .

[5]  Sergio Segura,et al.  Mutation testing on an object-oriented framework: An experience report , 2011, Inf. Softw. Technol..

[6]  A. Jefferson Offutt,et al.  A mutation carol: Past, present and future , 2011, Inf. Softw. Technol..

[7]  Mark Harman,et al.  An Analysis and Survey of the Development of Mutation Testing , 2011, IEEE Transactions on Software Engineering.

[8]  Andreas Zeller,et al.  The Impact of Equivalent Mutants , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[9]  John A. Clark,et al.  MESSI: Mutant Evaluation by Static Semantic Interpretation , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[10]  Anna Derezinska,et al.  A Quality Estimation of Mutation Clustering in C# Programs , 2013, DepCoS-RELCOMEX.

[11]  Mark Harman,et al.  Using program slicing to assist in the detection of equivalent mutants , 1999, Softw. Test. Verification Reliab..

[12]  Lionel C. Briand,et al.  Is mutation an appropriate tool for testing experiments? , 2005, ICSE.

[13]  J. A. Acree.  On mutation , 1980 .

[14]  A. Jefferson Offutt,et al.  Using compiler optimization techniques to detect equivalent mutants , 1994, Softw. Test. Verification Reliab..

[15]  John A. Clark,et al.  Investigating the effectiveness of object‐oriented testing strategies using the mutation method , 2001, Softw. Test. Verification Reliab..

[16]  Ilona Bluemke,et al.  Reduction of Computational Cost in Mutation Testing by Sampling Mutants , 2013, DepCoS-RELCOMEX.

[17]  Richard Torkar,et al.  Overcoming the Equivalent Mutant Problem: A Systematic Literature Review and a Comparative Experiment of Second Order Mutation , 2014, IEEE Transactions on Software Engineering.

[18]  A. Jefferson Offutt,et al.  Automatically detecting equivalent mutants and infeasible paths , 1997, Softw. Test. Verification Reliab..

[19]  John A. Clark,et al.  Investigating the effectiveness of object‐oriented testing strategies using the mutation method , 2001 .

[20]  Wynne Hsu,et al.  DESIGN OF MUTANT OPERATORS FOR THE C PROGRAMMING LANGUAGE , 2006 .

[21]  Andreas Zeller,et al.  (Un-)Covering Equivalent Mutants , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[22]  W. Eric Wong,et al.  Reducing the cost of mutation testing: An empirical study , 1995, J. Syst. Softw..

[23]  Anna Derezinska,et al.  Quality Evaluation of Object-Oriented and Standard Mutation Operators Applied to C# Programs , 2012, TOOLS.

[24]  Timothy Alan Budd,et al.  Mutation analysis of program test data , 1980 .

[25]  Gregg Rothermel,et al.  An experimental determination of sufficient mutant operators , 1996, TSEM.

[26]  S. N. Weiss,et al.  All-Uses versus Mutation Testing : An ExperimentalComparison of E ectiveness , 1996 .

[27]  A. Jefferson Offutt,et al.  Establishing Theoretical Minimal Sets of Mutants , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation.

[28]  Leonardo Bottaci,et al.  Efficiency of mutation operators and selective mutation strategies: an empirical study , 1999, Softw. Test. Verification Reliab..

[29]  Vincent Beroulle,et al.  Mutation sampling technique for the generation of structural test data , 2005, Design, Automation and Test in Europe.

[30]  A. Jefferson Offutt,et al.  A logic mutation approach to selective mutation for programs and queries , 2011, Inf. Softw. Technol..