Modeling and Testing Implementations of Protocols with Complex Messages

This paper presents a new language called APSL for formally describing protocols to facilitate automated testing. Many real world communication protocols exchange messages whose structures are not trivial, e.g. they may consist of multiple and nested fields, some could be optional, and some may have values that depend on other fields. To properly test implementations of such a protocol, it is not sufficient to only explore different orders of sending and receiving messages. We also need to investigate if the implementation indeed produces correctly formatted messages, and if it responds correctly when it receives different variations of every message type. APSL's main contribution is its sublanguage that is expressive enough to describe complex message formats, both text-based and binary. As an example, this paper also presents a case study where APSL is used to model and test a subset of Courier IMAP email server.

[1]  Jürgen Lind.  Specifying Agent Interaction Protocols with Standard UML , 2001, AOSE.

[2]  Hareton K. N. Leung,et al.  A survey of combinatorial testing , 2011, CSUR.

[3]  Stanislaw Budkowski,et al.  An Introduction to Estelle: A Specification Language for Distributed Systems , 1987, Comput. Networks.

[4]  Thierry Jéron,et al.  Using On-The-Fly Verification Techniques for the Generation of test Suites , 1996, CAV.

[5]  Tanja E. J. Vos,et al.  A Reinforcement Learning Approach to Automated GUI Robustness Testing , 2012 .

[6]  Graeme Smith,et al.  The Object-Z Specification Language , 1999, Advances in Formal Methods.

[7]  Gregor D. Scott.  Guide for Internet Standards Writers , 1998, RFC.

[8]  Jon Postel,et al.  Instructions to RFC Authors , 1997, RFC.

[9]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[10]  Arie Middelkoop,et al.  Functional Instrumentation of ActionScript Programs with Asil , 2011, IFL.

[11]  Lex Heerink,et al.  Formal Test Automation: The Conference Protocol with PHACT , 2000, TestCom.

[12]  Jan Tretmans,et al.  TorX: Automated Model-Based Testing , 2003 .

[13]  Mark R. Crispin.  Internet Message Access Protocol - Version 4rev1 , 1996, RFC.

[14]  A. Pol.  Clustering and Dynamic Invariant Detection , 2015 .

[15]  Koen Claessen,et al.  QuickCheck: a lightweight tool for random testing of Haskell programs , 2011, SIGP.

[16]  Dieter Hogrefe,et al.  An introduction to the testing and test control notation (TTCN-3) , 2003, Comput. Networks.

[17]  Nicolae Goga,et al.  Comparing TorX, Autolink, TGV and UIO Test Algorithms , 2001, SDL Forum.

[18]  I. S. Wishnu.  T3, a Combinator-Based Random Testing Tool for Java: Benchmarking , 2014 .

[19]  Phil McMinn,et al.  Search‐based software test data generation: a survey , 2004, Softw. Test. Verification Reliab..

[20]  Stephen McCamant,et al.  The Daikon system for dynamic detection of likely invariants , 2007, Sci. Comput. Program..

[21]  Dieter Hogrefe,et al.  The CCITT-Specification and Description Language SDL , 1989, Comput. Networks.

[22]  Tanja E. J. Vos,et al.  The FITTEST Tool Suite for Testing Future Internet Applications , 2013, FITTEST@ICTSS.

[23]  David L. Mills,et al.  Internet Engineering Task Force (ietf) Network Time Protocol Version 4: Protocol and Algorithms Specification , 2010 .

[24]  T. Tervoort.  A Protocol Specification Language for Testing Implementations , 2016 .

[25]  Jan Tretmans,et al.  A Formal Approach to Conformance Testing , 1993, Protocol Test Systems.

[26]  Martin Gogolla,et al.  Object Constraint Language (OCL): A Definitive Guide , 2012, SFM.

[27]  Gordon Fraser,et al.  EvoSuite: automatic test suite generation for object-oriented software , 2011, ESEC/FSE '11.

[28]  Larry L. Peterson,et al.  binpac: a yacc for writing application protocol parsers , 2006, IMC '06.