Test-Case Generation and Coverage Analysis for Nondeterministic Systems Using Model-Checkers

Nondeterminism is used as a means of under specification or implementation choice in specifications, and it is often necessary if part of a system or the environment is unpredictable. The use of model-checker counterexamples as test-cases is a popular technique in model-based testing. Even though model-checkers can handle nondeterministic models for verification purposes, the use of nondeterministic models for test-case generation is not directly possible. A counterexample is an example execution path where alternative paths might also be valid. Consequently, testing could falsely identify correct implementations as erroneous. This paper describes how to use model-checkers to derive test-cases from nondeterministic models by applying postprocessing to the counterexamples. The influence of nondeterminism on coverage measurement with model-checkers is analyzed, and known coverage criteria are adapted. This is useful for the execution of test-cases on nondeterministic systems, where special treatment is necessary.

[1]  Vadim Okun,et al.  Testing with Model Checker: Insuring Fault Visibility , 2002 .

[2]  G. Bochmann,et al.  Testing deterministic implementations from nondeterministic FSM specifications , 1996 .

[3]  Gordon Fraser,et al.  Property relevant software testing with model-checkers , 2006, SOEN.

[4]  Fred Kröger,et al.  Temporal Logic of Programs , 1987, EATCS Monographs on Theoretical Computer Science.

[5]  Fausto Giunchiglia,et al.  NUSMV: A New Symbolic Model Verifier , 1999, CAV.

[6]  S. Easterbrook,et al.  Generating Test Oracles via Model Checking , 1997 .

[7]  Insup Lee,et al.  Specification-based testing with linear temporal logic , 2004, Proceedings of the 2004 IEEE International Conference on Information Reuse and Integration, 2004. IRI 2004..

[8]  Angelo Gargantini,et al.  Using model checking to generate tests from requirements specifications , 1999, ESEC/FSE-7.

[9]  Paul E. Black.  Modeling and marshaling: making tests from model checker counterexamples , 2000, 19th DASC. 19th Digital Avionics Systems Conference. Proceedings (Cat. No.00CH37126).

[10]  Paul Ammann,et al.  A specification-based coverage metric to evaluate test sets , 1999, Proceedings 4th IEEE International Symposium on High-Assurance Systems Engineering.

[11]  Paul Ammann,et al.  Using model checking to generate tests from specifications , 1998, Proceedings Second International Conference on Formal Engineering Methods (Cat.No.98EX241).

[12]  Constance L. Heitmeyer,et al.  Model Checking Complete Requirements Specifications Using Abstraction , 2004, Automated Software Engineering.

[13]  Leonardo Mendonça de Moura,et al.  Generating efficient test sets with a model checker , 2004, Proceedings of the Second International Conference on Software Engineering and Formal Methods, 2004. SEFM 2004..

[14]  Thierry Jéron,et al.  TGV : theory , principles and algorithms A tool for the automatic synthesis of conformance test cases for non-deterministic reactive systems , 2004 .

[15]  Sanjai Rayadurgam,et al.  Coverage based test-case generation using model checkers , 2001, Proceedings. Eighth Annual IEEE International Conference and Workshop On the Engineering of Computer-Based Systems-ECBS 2001.

[16]  Wei Ding,et al.  Using a model checker to test safety properties , 2001, Proceedings Seventh IEEE International Conference on Engineering of Complex Computer Systems.