Industrial and Academic Protocol Testing: the Gap and the Means of Convergence

Academic research has recently made significant advances in the generation of test sequences from formal specifications and in the development of computer-aided test tools, with the aims of improving the effectiveness of testing. The industry, with its own method of testing, is shipping to the customer communication software that still has bugs, even after it has been tested in the factory. It could therefore be expected that academic testing methods would be used to help improve its effectiveness. Surprisingly, these methods are seldom used in the industry. This paper describes the gap between the two classes of methods, the reasons why academic methods are not widely adopted by the industry, an assessment of industrial and academic methods, and the ways in which they can converge. It is intended to open up a discussion on these rarely addressed topics.

[1]  Colin H. West The first ten years, the next ten years , 1990, PSTV.

[2]  Behçet Sarikaya,et al.  Test Generation from LOTOS Specifications , 1991, IEEE Trans. Computers.

[3]  Fabrizio Lombardi,et al.  Protocol Conformance Testing by Discriminating UIO Sequences , 1991, PSTV.

[4]  Robin Milner,et al.  A Calculus of Communicating Systems , 1980, Lecture Notes in Computer Science.

[5]  Richard Lai,et al.  Teaching Protocol Engineering in Honours Year , 1992, CSEE.

[6]  Michel Barbeau,et al.  FTAM test design using an automated test tool , 1989, IEEE INFOCOM '89, Proceedings of the Eighth Annual Joint Conference of the IEEE Computer and Communications Societies.

[7]  Alfred V. Aho,et al.  An optimization technique for protocol conformance test generation based on UIO sequences and rural Chinese postman tours , 1991, IEEE Trans. Commun..

[8]  Guney Gonenc,et al.  A Method for the Design of Fault Detection Experiments , 1970, IEEE Transactions on Computers.

[9]  Deepinder P. Sidhu,et al.  Formal Methods for Protocol Testing: A Detailed Study , 1989, IEEE Trans. Software Eng..

[10]  Jai-Yong Lee,et al.  A Well-Defined Estelle Specification for the Automatic Test Generation , 1991, IEEE Trans. Computers.

[11]  Erik Kwast Towards Automatic Test Generation for Protocol Data Aspects , 1991, PSTV.

[12]  Deepinder P. Sidhu,et al.  Probabilistic testing of protocols , 1989, SIGCOMM 1989.

[13]  Michiel van Genuchten,et al.  Why is Software Late? An Empirical Study of Reasons For Delay in Software Development , 1991, IEEE Trans. Software Eng..

[14]  Deepinder P. Sidhu,et al.  Protocol testing: the first ten years, the next ten years , 1990 .

[15]  Carl A. Sunshine,et al.  Protocol Specification, Testing and Verification - Guest Editorial , 1982, Comput. Networks.

[16]  Ferhat Khendek,et al.  Test Selection Based on Finite State Models , 1991, IEEE Trans. Software Eng..

[17]  Susan A. Sherer,et al.  A cost-effective approach to testing , 1991, IEEE Software.

[18]  Deepinder P. Sidhu,et al.  Fault coverage of probabilistic test sequences , 1990 .

[19]  Krishan K. Sabnani,et al.  Algorithmic generation of protocol conformance tests , 1990, AT&T Technical Journal.

[20]  Hugh M. Pattinson,et al.  Information technology and telecommunications , 1995 .