FSM-based testing from user defined faults adapted to incremental and mutation testing

We study the problem of deriving a test suite with guaranteed fault coverage from a given finite state machine specification with respect to some given user defined faults. We consider the case when an implementation under test can have more states than its specification while user defined faults are implemented in an arbitrary way. We show that our approach can be used for FSM-based incremental and mutation testing and correspondingly we investigate cases that can be used for reducing length of obtained test suites. In some cases, worst-case length of obtained test suite becomes polynomial. Experiments show significant gains is using our approach in comparison to testing the whole specification.

[1]  Simone do Rocio Senger de Souza,et al.  Mutation Testing Applied to Estelle Specifications , 2004, Software Quality Journal.

[2]  Mark Harman,et al.  An Analysis and Survey of the Development of Mutation Testing , 2011, IEEE Transactions on Software Engineering.

[3]  Nina Yevtushenko,et al.  An Improved Conformance Testing Method , 2005, FORTE.

[4]  Gregor von Bochmann,et al.  FSM-based incremental conformance testing methods , 2004, IEEE Transactions on Software Engineering.

[5]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[6]  Ana R. Cavalli,et al.  Experimental evaluation of FSM-based testing methods , 2005, Third IEEE International Conference on Software Engineering and Formal Methods (SEFM'05).

[7]  Nina Yevtushenko,et al.  Test Generation Driven by User-defined Fault Models , 1999, IWTCS.

[8]  Ferhat Khendek,et al.  Test Selection Based on Finite State Models , 1991, IEEE Trans. Software Eng..

[9]  W. Eric Wong,et al.  Mutation Testing Applied to Validate SDL Specifications , 2004, TestCom.

[10]  Márcio Eduardo Delamaro,et al.  Proteum/FSM: a tool to support finite state machine validation based on mutation testing , 1999, Proceedings. SCCC'99 XIX International Conference of the Chilean Computer Science Society.

[11]  Robert M. Hierons,et al.  Mutation testing from probabilistic and stochastic finite state machines , 2009, J. Syst. Softw..

[12]  Arthur Gill,et al.  Introduction to the theory of finite-state machines , 1962 .

[13]  Nina Yevtushenko,et al.  Generating Reduced Tests for FSMs with Extra States , 2009, TestCom/FATES.

[14]  Jin-hua Li,et al.  Mutation Analysis for Testing Finite State Machines , 2009, 2009 Second International Symposium on Electronic Commerce and Security.

[15]  M. P. Vasilevskii.  Failure diagnosis of automata , 1973 .

[16]  Alexandre Petrenko,et al.  Protocol testing: review of methods and relevance for software testing , 1994, ISSTA '94.

[17]  Ana R. Cavalli,et al.  FSM-based conformance testing methods: A survey annotated with experimental evaluation , 2010, Inf. Softw. Technol..

[18]  David Lee,et al.  Principles and methods of testing finite state machines-a survey , 1996, Proc. IEEE.

[19]  Alexandre Petrenko.  Checking Experiments with Protocol Machines , 1991, Protocol Test Systems.

[20]  David Lee,et al.  Testing Finite-State Machines: State Identification and Verification , 1994, IEEE Trans. Computers.

[21]  Nina Yevtushenko,et al.  Test Suite Generation from a FSM with a Given Type of Implementation Errors , 1992, PSTV.

[22]  Nina Yevtushenko,et al.  Testing from partial deterministic FSM specifications , 2005, IEEE Transactions on Computers.

[23]  David Lee,et al.  Testing Finite State Machines: Fault Detection , 1995, J. Comput. Syst. Sci..

[24]  Nina Yevtushenko,et al.  Nondeterministic State Machines in Protocol Conformance Testing , 1993, Protocol Test Systems.