Efficient Refinement Checking for Model-Based Mutation Testing

In model-based mutation testing, a test model is mutated for test case generation. The resulting test cases are able to detect whether the faults in the mutated models have been implemented in the system under test. For this purpose, a conformance check between the original and the mutated model is required. We have developed an approach for conformance checking of action systems, which are well-suited to specify reactive and non-deterministic systems. We rely on constraint solving techniques. Both, the conformance relation and the transition relation are encoded as constraint satisfaction problems. Earlier results showed the potential of our constraint-based approach to outperform explicit conformance checking techniques, which often face state space explosion. In this work, we go one step further and show optimisations that really boost our performance. In our experiments, we could reduce our runtimes by 80%.

[1]  Bruno Legeard,et al.  A taxonomy of model‐based testing approaches , 2012, Softw. Test. Verification Reliab..

[2]  Andrew William Roscoe,et al.  Model-checking CSP , 1994 .

[3]  Arnaud Gotlieb,et al.  Automatic test data generation using constraint solving techniques , 1998, ISSTA '98.

[4]  Bernhard K. Aichernig,et al.  Generating Distinguishing Tests Using the Minion Constraint Solver , 2010, 2010 Third International Conference on Software Testing, Verification, and Validation Workshops.

[5]  Bernhard K. Aichernig,et al.  Towards Symbolic Model-Based Mutation Testing: Combining Reachability and Refinement Checking , 2012, MBT.

[6]  Bernhard K. Aichernig,et al.  Towards Symbolic Model-Based Mutation Testing: Pitfalls in Expressing Semantics as Constraints , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[7]  Kaisa Sere,et al.  An Approach to Object-Orientation in Action Systems , 1998, MPC.

[8]  Mats Carlsson,et al.  An Open-Ended Finite Domain Constraint Solver , 1997, PLILP.

[9]  Bernhard K. Aichernig,et al.  Efficient Mutation Killers in Action , 2011, 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation.

[10]  Bernhard K. Aichernig,et al.  Automated Conformance Verification of Hybrid Systems , 2010, 2010 10th International Conference on Quality Software.

[11]  Timothy A. Budd,et al.  Program Testing by Specification Mutation , 1985, Comput. Lang..

[12]  Philip Alan Stocks,et al.  Applying formal methods to software testing , 1993 .

[13]  Bernhard K. Aichernig,et al.  Protocol Conformance Testing a SIP Registrar: an Industrial Application of Formal Methods , 2007, Fifth IEEE International Conference on Software Engineering and Formal Methods (SEFM 2007).

[14]  Ralph-Johan Back,et al.  Distributed cooperation with action systems , 1988, TOPL.

[15]  Bernard Botella,et al.  A CLP Framework for Computing Structural Test Data , 2000, Computational Logic.

[16]  Mark Harman,et al.  An Analysis and Survey of the Development of Mutation Testing , 2011, IEEE Transactions on Software Engineering.

[17]  Kaisa Sere,et al.  Stepwise Refinement of Action Systems , 1991, Struct. Program..

[18]  Ralph-Johan Back,et al.  Decentralization of process nets with centralized control , 1983, PODC '83.

[19]  Bernhard K. Aichernig,et al.  Mutation testing in UTP , 2009, Formal Aspects of Computing.

[20]  Richard G. Hamlet,et al.  Testing Programs with the Aid of a Compiler , 1977, IEEE Transactions on Software Engineering.

[21]  Jan Tretmans,et al.  Test Generation with Inputs, Outputs and Repetitive Quiescence , 1996, Softw. Concepts Tools.

[22]  Margo McCall,et al.  IEEE Computer Society , 2019, Encyclopedia of Software Engineering.

[23]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[24]  Augusto Sampaio,et al.  Guided Test Generation from CSP Models , 2008, ICTAC.

[25]  Takeo Kanade,et al.  Unifying Theories of Programming , 2010, Lecture Notes in Computer Science.

[26]  Vadim Okun,et al.  Testing with Model Checker: Insuring Fault Visibility , 2002 .

[27]  Bowen Alpern,et al.  Detecting equality of variables in programs , 1988, POPL '88.

[28]  Alexandre Petrenko,et al.  Can a Model Checker Generate Tests for Non-Deterministic Systems? , 2007, MBT.

[29]  Paul Ammann,et al.  Using model checking to generate tests from specifications , 1998, Proceedings Second International Conference on Formal Engineering Methods (Cat.No.98EX241).