Finite State Machine Based Formal Methods in Protocol Conformance Testing: From Theory to Implementation

Abstract Recent evolution of communication networks has led toward increasingly complex communication protocols to interconnect heterogeneous systems. In order for such systems to function properly, communication protocols require formal methodologies for verification, implementation and testing. In this paper, the use of formal methods for protocol conformance testing is presented. Four major formal techniques for conformance test generation reported in the literature are compared: transition tours, distinguishing sequences, characterizing sequences and unique input/output sequences. The implementation of each approach as a test generation methodology is illustrated. The result is the capability to generate abstract test suites that require a minimum number of steps to execute while maintaining complete coverage of the state transitions (or test purposes) of the protocol. The impact of the formal methodologies on the practical aspects of conformance testing, including the test system implementation and automatic test script generation, is discussed. The experience based on testing various protocol implementations from multiple manufacturers at AT&T Bell Laboratories is described. The protocols tested include X.25 and ISDN Q.921 and Q.931 for basic and primary rate interfaces. In order to standardize various efforts on conformance testing, the ISO and the CCITT have developed principles to specify abstract and executable test suites, and methodologies to design testbed architectures. The relationship between such standards and the formal methods for test generation is explored.

[1]  John R. Pavel,et al.  Some Experiences of Testing Protocol Implementations , 1984, PSTV.

[2]  Luigi Logrippo,et al.  Derivation of Test Cases for LAP-B from a LOTOS Specification , 1989, FORTE.

[3]  Hasan Ural.  A Test Derivation Method for Protocol Conformance Testing , 1987, PSTV.

[4]  Krishan K. Sabnani,et al.  A new technique for generating protocol test , 1985, SIGCOMM 1985.

[5]  Richard M. Karp,et al.  Theoretical Improvements in Algorithmic Efficiency for Network Flow Problems , 1972, Combinatorial Optimization.

[6]  Robert L. Probert,et al.  User-Guided Test Sequence Generation , 1983, Protocol Specification, Testing, and Verification.

[7]  Behcet Sarikaya,et al.  A computer-aided design tool for protocol testing , 1988, IEEE INFOCOM '88,Seventh Annual Joint Conference of the IEEE Computer and Communcations Societies. Networks: Evolution or Revolution?.

[8]  Edward P. Hsieh,et al.  Checking Experiments ror Sequential Machines , 1971, IEEE Transactions on Computers.

[9]  G. Hoover,et al.  X.25 Conformance testing-A tutorial , 1986, IEEE Communications Magazine.

[10]  D. Rayner.  OSI Conformance Testing , 1987, Comput. Networks.

[11]  Parag K. Lala,et al.  Fault tolerant and fault testable hardware design , 1985 .

[12]  Gregor von Bochmann,et al.  A Survey of Formal Methods , 1982 .

[13]  F. C. Hennine.  Fault detecting experiments for sequential circuits , 1964, SWCT 1964.

[14]  Krishan K. Sabnani,et al.  A Protocol Test Generation Procedure , 1988, Comput. Networks.

[15]  Zvi Kohavi,et al.  Checking experiments for sequential machines , 1974, Inf. Sci..

[16]  Robert E. Tarjan,et al.  Data structures and network algorithms , 1983, CBMS-NSF regional conference series in applied mathematics.

[17]  Jack Edmonds,et al.  Matching, Euler tours and the Chinese postman , 1973, Math. Program..

[18]  R.J. Linn,et al.  Testing OSI protocols at the national bureau of standards , 1983, Proceedings of the IEEE.

[19]  Güney Gönenç,et al.  A Method for the Design of Fault Detection Experiments , 1970 .

[20]  Jan Karel Lenstra,et al.  On general routing problems , 1976, Networks.

[21]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[22]  Alfred V. Aho,et al.  An optimization technique for protocol conformance test generation based on UIO sequences and rural Chinese postman tours , 1991, IEEE Trans. Commun..

[23]  Behcet Sarikaya.  Test design for computer network protocols , 1984 .

[24]  Gregor von Bochmann,et al.  A Test Design Methodology for Protocol Testing , 1987, IEEE Transactions on Software Engineering.

[25]  Gregor von Bochmann,et al.  Synchronization and Specification Issues in Protocol Testing , 1984, IEEE Trans. Commun..

[26]  Deepinder Sidhu,et al.  Fault coverage of protocol test methods , 1988, IEEE INFOCOM '88,Seventh Annual Joint Conference of the IEEE Computer and Communcations Societies. Networks: Evolution or Revolution?.