Using conditional mutation to increase the efficiency of mutation analysis

Assessing testing strategies and test sets is a crucial part of software testing. Mutation analysis is, among other approaches, a suitable technique for this purpose. However, compared with other methods it is rather time-consuming and applying mutation analysis to large software systems is still problematic. This paper presents a versatile approach, called conditional mutation, which increases the efficiency of mutation analysis. This new method significantly reduces the time overhead for generating and executing the mutants. Results are reported for eight investigated programs up to 373,000 lines of code and 406,000 generated mutants. Furthermore, conditional mutation has been integrated into the Java 6 Standard Edition compiler. Thus, it is widely applicable and not limited to a certain testing tool or framework.

[1]  Timothy Alan Budd,et al.  Mutation analysis of program test data , 1980 .

[2]  A. Jefferson Offutt,et al.  Mutation analysis using mutant schemata , 1993, ISSTA '93.

[3]  Gregg Rothermel,et al.  An experimental determination of sufficient mutant operators , 1996, TSEM.

[4]  Robert V. Binder,et al.  Testing Object-Oriented Systems: Models, Patterns, and Tools , 1999 .

[5]  William E. Howden,et al.  Weak Mutation Testing and Completeness of Test Sets , 1982, IEEE Transactions on Software Engineering.

[6]  Richard A. DeMillo,et al.  Compiler-integrated program mutation , 1991, [1991] Proceedings The Fifteenth Annual International Computer Software & Applications Conference.

[7]  Jeffrey M. Voas,et al.  PIE: A Dynamic Failure-Based Technique , 1992, IEEE Trans. Software Eng..

[8]  Byoungju Choi,et al.  PMothra: scheduling mutants for execution on a hypercube , 1989, TAV3.

[9]  Mark Harman,et al.  An Analysis and Survey of the Development of Mutation Testing , 2011, IEEE Transactions on Software Engineering.

[10]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[11]  A. Jefferson Offutt,et al.  Mutation 2000: uniting the orthogonal , 2001 .

[12]  Akbar Siami Namin,et al.  Sufficient mutation operators for measuring test effectiveness , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[13]  K. N. King,et al.  A fortran language system for mutation‐based software testing , 1991, Softw. Pract. Exp..

[14]  A. Jefferson Offutt,et al.  Investigations of the software testing coupling effect , 1992, TSEM.

[15]  A. Jefferson Offutt,et al.  MuJava: a mutation system for java , 2006, ICSE.

[16]  A. Jefferson Offutt,et al.  Automatically detecting equivalent mutants and infeasible paths , 1997, Softw. Test. Verification Reliab..

[17]  Lionel C. Briand,et al.  Is mutation an appropriate tool for testing experiments? , 2005, ICSE.

[18]  Mark Harman,et al.  Higher Order Mutation Testing , 2009, Inf. Softw. Technol..

[19]  A. Jefferson Offutt,et al.  Automatically detecting equivalent mutants and infeasible paths , 1997 .

[20]  Antonia Bertolino,et al.  Software Testing Research: Achievements, Challenges, Dreams , 2007, Future of Software Engineering (FOSE '07).

[21]  Andreas Zeller,et al.  (Un-)Covering Equivalent Mutants , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.