Iterative automatic test generation method for telecommunication protocols

Standardized languages used for protocol specification provide an excellent basis for both automatic and manual test generation. Test generation is composed of two steps, the derivation of test cases from the specification, and the selection of the test cases to be included in the final test suite in order to reduce its execution time. This article proposes a new method that aims to decrease the total number of test cases generated automatically by a test derivation algorithm and at the same time to reduce the computation requirements for the test selection procedure. It creates an iteration cycle on the model of evolutionary algorithms, where the test derivation and selection are done simultaneously. In each cycle first a ''small'' test suite is derived then optimized, evaluated and finally compared to the best suite so far. It is kept as the best suite, if it is found better according to some well-defined evaluation criteria and test suite metrics. This iteration condition is based on the test selection criteria. The article presents an experiment where iterative algorithms are compared to two simple test derivation methods from different aspects.

[1]  G. Bochmann,et al.  Fault Models in Testing , 1991, Protocol Test Systems.

[2]  Itu-T.  Specification and Description Language (SDL) , 1999 .

[3]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[4]  Nicolae Goga,et al.  Formal Test Automation: A Simple Experiment , 1999, IWTCS.

[5]  Thomas Bäck,et al.  An Overview of Evolutionary Computation , 1993, ECML.

[6]  Pavel Brazdil,et al.  Proceedings of the European Conference on Machine Learning , 1993 .

[7]  Simone do Rocio Senger de Souza,et al.  Mutation Testing Applied to Estelle Specifications , 2004, Software Quality Journal.

[8]  Gyula Csopaki,et al.  Applying Mutation Analysis to SDL Specifications , 2003, SDL Forum.

[9]  Balázs Kotnyek,et al.  Application of heuristic methods for conformance test selection , 2002, Eur. J. Oper. Res..

[10]  Deepinder Sidhu,et al.  Fault coverage of protocol test methods , 1988, IEEE INFOCOM '88,Seventh Annual Joint Conference of the IEEE Computer and Communcations Societies. Networks: Evolution or Revolution?.

[11]  Nicolae Goga,et al.  Test Selection, Trace Distance and Heuristics , 2002, TestCom.

[12]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[13]  Diego Latella,et al.  Formal methods for distributed system development : FORTE/PSTV 2000, IFIP TC6 WG6.1 Joint International Conference on Formal Description Techniques for Distributed Systems and Communication Protocols (FORTE XIII) and Protocol Specification, Testing and Verification (PSTV XX), October 10-13, 2000, Pi , 2000 .

[14]  W. Eric Wong,et al.  Mutation Testing Applied to Validate SDL Specifications , 2004, TestCom.

[15]  D. Richard Kuhn.  A Technique for Analyzing the Effects of Changes in Formal Specifications , 1992, Comput. J..

[16]  Dieter Hogrefe,et al.  SDL : formal object-oriented language for communicating systems , 1997 .

[17]  Enrico Vicario,et al.  Specification and Simulation of Real Time Concurrent Systems Using Standard SDL Tools , 2003, SDL Forum.

[18]  Robert L. Probert,et al.  Formulation of the Interaction Test Coverage Problem as an Integer Program , 2002, TestCom.

[19]  David Lee,et al.  Principles and methods of testing finite state machines-a survey , 1996, Proc. IEEE.

[20]  T. Csondes,et al.  A mathematical programming method in test selection , 1997, Proceedings 23rd Euromicro Conference New Frontiers of Information Technology - Short Contributions -.

[21]  Zoltán Pap,et al.  Automatic Test Selection based on CEFSM Specifications , 2002, Acta Cybern..