Formally transforming user-model testing problems into implementer-model testing problems and viceversa

Abstract There are several methods to assess the capability of a test suite to detect faults in a potentially wrong system. We explore two methods based on considering some probabilistic information. In the first one, we assume that we are provided with a probabilistic user model . This is a model denoting the probability that the entity interacting with the system takes each available choice. In the second one, we suppose that we have a probabilistic implementer model , that is, a model denoting the probability that the implementer makes each possible fault while constructing the system. We show that both testing scenarios are strongly related. In particular, we prove that any user can be translated into an implementer model in such a way that the optimality of tests is preserved, that is, a test suite is optimal for the user if and only if it is optimal for the resulting implementer. Another translation, working in the opposite direction, fulfills the reciprocal property. Thus, we conclude that any test selection criterium designed for one of these testing problems can be used for the other one, once the model has been properly translated. Besides, the applicability of user models to other kinds of testing approaches is considered.

[1]  Rachel Cardell-Oliver Conformance Tests for Real-Time Systems with Timed Automata Specifications , 2000, Formal Aspects of Computing.

[2]  Jan Tretmans,et al.  Model Based Testing with Labelled Transition Systems , 2008, Formal Methods and Testing.

[3]  Mercedes G. Merayo,et al.  HOTL: Hypotheses and observations testing logic , 2007, J. Log. Algebraic Methods Program..

[4]  Alexandre Petrenko,et al.  Fault Model-Driven Test Derivation from Finite State Models: Annotated Bibliography , 2000, MOVEP.

[5]  Fabio Paternò,et al.  CTTE: Support for Developing and Analyzing Task Models for Interactive System Design , 2002, IEEE Trans. Software Eng..

[6]  Simeon C. Ntafos,et al.  A Comparison of Some Structural Testing Strategies , 1988, IEEE Trans. Software Eng..

[7]  Richard G. Hamlet,et al.  Testing Programs with the Aid of a Compiler , 1977, IEEE Transactions on Software Engineering.

[8]  R. Hierons,et al.  Mutation Testing from Probabilistic Finite State Machines , 2007, Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007).

[9]  Mark Harman,et al.  Using program slicing to assist in the detection of equivalent mutants , 1999, Softw. Test. Verification Reliab..

[10]  Alexandre Petrenko,et al.  Test Selection Based on Communicating Nondeterministic Finite-State Machines Using a Generalized WP-Method , 1994, IEEE Trans. Software Eng..

[11]  Ferhat Khendek,et al.  Test Selection Based on Finite State Models , 1991, IEEE Trans. Software Eng..

[12]  Laurence Nigay,et al.  Formal Testing of Multimodal Interactive Systems , 2008, EHCI/DS-VIS.

[13]  Leonardo Bottaci,et al.  Efficiency of mutation operators and selective mutation strategies: an empirical study , 1999 .

[14]  Nina Yevtushenko,et al.  Fault Models for Testing in Context , 1996, FORTE.

[15]  Ismael Rodríguez,et al.  Formally comparing user and implementer model-based testing methods , 2008, 2008 IEEE International Conference on Software Testing Verification and Validation Workshop.

[16]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[17]  Manuel Núñez,et al.  Customized Testing for Probabilistic Systems , 2006, TestCom.

[18]  Ana R. Cavalli,et al.  A passive testing approach based on invariants: application to the WAP , 2005, Comput. Networks.

[19]  Jan Tretmans,et al.  A Formal Approach to Conformance Testing , 1993, Protocol Test Systems.

[20]  Laurence Nigay,et al.  Synchronous testing of multimodal systems: an operational profile-based approach , 2005, 16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05).

[21]  Ioannis Parissis,et al.  Automated Test of Interactive Applications Using Task Trees , 2008, 2008 IEEE International Conference on Software Testing Verification and Validation Workshop.

[22]  David Lee,et al.  Principles and methods of testing finite state machines-a survey , 1996, Proc. IEEE.

[23]  Robert M. Hierons Comparing test sets and criteria in the presence of test hypotheses and fault domains , 2002, TSEM.

[24]  Jan Tretmans,et al.  Test Generation with Inputs, Outputs and Repetitive Quiescence , 1996, Softw. Concepts Tools.

[25]  Manuel Núñez,et al.  Derivation of a Suitable Finite Test Suite for Customized Probabilistic Systems , 2006, FORTE.