From behaviour preservation to behaviour modification: constraint-based mutant generation

The efficacy of mutation analysis depends heavily on its capability to mutate programs in such a way that they remain executable and exhibit deviating behaviour. Whereas the former requires knowledge about the syntax and static semantics of the programming language, the latter requires some least understanding of its dynamic semantics, i.e., how expressions are evaluated. We present an approach that is knowledgeable enough to generate only mutants that are both syntactically and semantically correct and likely exhibit non-equivalent behaviour. Our approach builds on our own prior work on constraint-based refactoring tools, and works by negating behaviour-preserving constraints. As a proof of concept we present an enhanced implementation of the Access Modifier Change operator for Java programs whose naive implementations create huge numbers of mutants that do not compile or leave behaviour unaltered. While we cannot guarantee that our generated mutants are non-equivalent, we can demonstrate a considerable reduction in the number of vain mutant generations, leading to substantial temporal savings.

[1]  Friedrich Steimann,et al.  From Public to Private to Absent: Refactoring Java Programs under Constrained Accessibility , 2009, ECOOP.

[2]  A. Jefferson Offutt,et al.  The class-level mutants of MuJava , 2006, AST '06.

[3]  James Gosling,et al.  The Java Language Specification, 3rd Edition , 2005 .

[4]  Guy L. Steele,et al.  Java(TM) Language Specification, The (3rd Edition) (Java (Addison-Wesley)) , 2005 .

[5]  A. Jefferson Offutt,et al.  Automatically detecting equivalent mutants and infeasible paths , 1997, Softw. Test. Verification Reliab..

[6]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[7]  Douglas Baldwin,et al.  Heuristics for Determining Equivalence of Program Mutations. , 1979 .

[8]  Friedrich Steimann,et al.  Systematically refactoring inheritance to delegation in java , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[9]  Torbjörn Ekman,et al.  Sound and extensible renaming for java , 2008, OOPSLA.

[10]  Guy L. Steele,et al.  The Java Language Specification , 1996 .

[11]  A. Jefferson Offutt,et al.  Inter-class mutation operators for Java , 2002, 13th International Symposium on Software Reliability Engineering, 2002. Proceedings..

[12]  Frank Tip,et al.  Refactoring using type constraints , 2011, TOPL.

[13]  William E. Howden,et al.  Weak Mutation Testing and Completeness of Test Sets , 1982, IEEE Transactions on Software Engineering.

[14]  A. Jefferson Offutt,et al.  Using compiler optimization techniques to detect equivalent mutants , 1994, Softw. Test. Verification Reliab..

[15]  A. Jefferson Offutt,et al.  Detecting equivalent mutants and the feasible path problem , 1996, Proceedings of 11th Annual Conference on Computer Assurance. COMPASS '96.

[16]  Dana Angluin,et al.  Two notions of correctness and their relation to testing , 1982, Acta Informatica.

[17]  Mark Harman,et al.  The relationship between program dependence and mutation analysis , 2001 .

[18]  A. Jefferson Offutt,et al.  Constraint-Based Automatic Test Data Generation , 1991, IEEE Trans. Software Eng..

[19]  Richard G. Hamlet,et al.  Testing Programs with the Aid of a Compiler , 1977, IEEE Transactions on Software Engineering.

[20]  Mark Harman,et al.  Using program slicing to assist in the detection of equivalent mutants , 1999, Softw. Test. Verification Reliab..

[21]  Frank Tip,et al.  Refactoring for generalization using type constraints , 2003, OOPSLA '03.

[22]  Andreas Zeller,et al.  Efficient mutation testing by checking invariant violations , 2009, ISSTA.