Efficient Mutation Killers in Action

This paper presents the techniques and results of a novel model-based test case generation approach that automatically derives test cases from UML state machines. Mutation testing is applied on the modeling level to generate test cases. We present the test case generation approach, discuss the tool chain, and present the properties of the generated test cases. The main contribution of this paper is an empirical study of a car alarm system where different strategies for killing mutants are compared. We present detailed figures on the effectiveness of the test case generation technique. Although UML serves as an input language, all techniques are grounded on solid foundations: we give UML state transition diagrams a formal semantics by mapping them to Back's action systems.

[1]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[2]  Richard G. Hamlet,et al.  Testing Programs with the Aid of a Compiler , 1977, IEEE Transactions on Software Engineering.

[3]  Jan Tretmans,et al.  Test Generation with Inputs, Outputs and Repetitive Quiescence , 1996, Softw. Concepts Tools.

[4]  Kim G. Larsen,et al.  Testing Real-Time Systems Using UPPAAL , 2008, Formal Methods and Testing.

[5]  Bernhard K. Aichernig,et al.  Mapping UML to Labeled Transition Systems for Test-Case Generation - A Translation via Object-Oriented Action Systems , 2009, FMCO.

[6]  Paul Ammann,et al.  Using model checking to generate tests from specifications , 1998, Proceedings Second International Conference on Formal Engineering Methods (Cat.No.98EX241).

[7]  Ralph-Johan Back,et al.  Decentralization of Process Nets with Centralized Control , 1983, PODC.

[8]  Marie-Claude Gaudel,et al.  Software testing based on formal specifications: a theory and a tool , 1991, Softw. Eng. J..

[9]  Paulo César Masiero,et al.  Mutation testing applied to validate specifications based on statecharts , 1999, Proceedings 10th International Symposium on Software Reliability Engineering (Cat. No.PR00443).

[10]  Philip Alan Stocks,et al.  Applying formal methods to software testing , 1993 .

[11]  Bernhard K. Aichernig,et al.  UML in action: a two-layered interpretation for testing , 2011, SOEN.

[12]  Bernhard K. Aichernig,et al.  Protocol Conformance Testing a SIP Registrar: an Industrial Application of Formal Methods , 2007, Fifth IEEE International Conference on Software Engineering and Formal Methods (SEFM 2007).

[13]  Massimiliano Di Penta,et al.  Assessing and improving state-based class testing: a series of experiments , 2004, IEEE Transactions on Software Engineering.

[14]  Timothy A. Budd,et al.  Program Testing by Specification Mutation , 1985, Comput. Lang..

[15]  A. Jefferson Offutt,et al.  MuJava: a mutation system for java , 2006, ICSE.

[16]  R. Hierons,et al.  Mutation Testing from Probabilistic Finite State Machines , 2007, Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007).

[17]  Peter Fröhlich,et al.  Automated Test Case Generation from Dynamic Models , 2000, ECOOP.

[18]  Diego Latella,et al.  Formal test-case generation for UML statecharts , 2004, Proceedings. Ninth IEEE International Conference on Engineering of Complex Computer Systems.

[19]  Bernhard K. Aichernig,et al.  Automated Conformance Verification of Hybrid Systems , 2010, 2010 10th International Conference on Quality Software.

[20]  Dirk Seifert,et al.  Test Case Generation for UML Statecharts , 2003, Ershov Memorial Conference.

[21]  Bernhard K. Aichernig,et al.  Model-Based Mutation Testing of Hybrid Systems , 2009, FMCO.

[22]  Thierry Jéron,et al.  TGV : theory , principles and algorithms A tool for the automatic synthesis of conformance test cases for non-deterministic reactive systems , 2004 .