Timed implementation relations for the distributed test architecture

In order to test systems that have physically distributed interfaces, called ports, we might use a distributed approach in which there is a separate tester at each port. If the testers do not synchronise during testing then we cannot always determine the relative order of events observed at different ports and this leads to new notions of correctness that have been described using corresponding implementation relations. We study the situation in which each tester has a local clock and timestamps its observations. If we know nothing about how the local clocks relate then this does not affect the implementation relation while if the local clocks agree exactly then we can reconstruct the sequence of observations made. In practice, however, we are likely to be between these extremes: the local clocks will not agree exactly but we have some information regarding how they can differ. We start by assuming that a local tester interacts synchronously with the corresponding port of the system under test and then extend this to the case where communications can be asynchronous, considering both the first-in-first-out (FIFO) case and the non-FIFO case. The new implementation relations are stronger than implementation relations for distributed testing that do not use timestamps but still reflect the distributed nature of observations. This paper explores these alternatives and derives corresponding implementation relations.

[1]  Hasan Ural,et al.  Synchronizable test sequences based on multiple UIO sequences , 1995, TNET.

[2]  Robert M. Hierons,et al.  Controllable Test Cases for the Distributed Test Architecture , 2008, ATVA.

[3]  A. Jefferson Offutt,et al.  Introduction to Software Testing , 2008 .

[4]  Robert M. Hierons Oracles for Distributed Testing , 2012, IEEE Transactions on Software Engineering.

[5]  Craig P. L Williams Checking sequences for distributed test architectures , 2003 .

[6]  Matthew Hennessy,et al.  Testing Theories for Asynchronous Languages , 1998, FSTTCS.

[7]  Rocco De Nicola,et al.  A Theory of "May" Testing for Asynchronous Languages , 1999, FoSSaCS.

[8]  Leslie Lamport,et al.  Time, clocks, and the ordering of events in a distributed system , 1978, CACM.

[9]  Alan Hartman,et al.  Using a model-based test generator to test for standard conformance , 2002, IBM Syst. J..

[10]  Nina Yevtushenko,et al.  Testing from partial deterministic FSM specifications , 2005, IEEE Transactions on Computers.

[11]  Yves Le Traon,et al.  Efficient Distributed Test Architectures for Large-Scale Systems , 2010, ICTSS.

[12]  Guney Gonenc,et al.  A Method for the Design of Fault Detection Experiments , 1970, IEEE Transactions on Computers.

[13]  Wolfgang Grieskamp,et al.  Model‐based quality assurance of protocol documentation: tools and methodology , 2011, Softw. Test. Verification Reliab..

[14]  Myungchul Kim,et al.  An enhanced model for testing asynchronous communicating systems , 1999, FORTE.

[15]  Paul Gastin,et al.  A Fresh Look at Testing for Asynchronous Communication , 2006, ATVA.

[16]  David Lee,et al.  Principles and methods of testing finite state machines-a survey , 1996, Proc. IEEE.

[17]  Robert M. Hierons,et al.  Implementation relations and test generation for systems with distributed interfaces , 2011, Distributed Computing.

[18]  Hasan Ural,et al.  The Synchronization Problem in Protocol Testing and its Complexity , 1991, Inf. Process. Lett..

[19]  Felix Naumann,et al.  Data fusion , 2009, CSUR.

[20]  Thierry Jéron,et al.  Remote testin can be as powerful as local testing , 1999, FORTE.

[21]  Boaz Patt-Shamir,et al.  Optimal and efficient clock synchronization under drifting clocks , 1999, PODC '99.

[22]  Omar Rafiq,et al.  Controllability and observability in distributed testing , 1999, Inf. Softw. Technol..

[23]  Jan Tretmans,et al.  Model Based Testing with Labelled Transition Systems , 2008, Formal Methods and Testing.

[24]  Omar Rafiq,et al.  Coordination Algorithm for Distributed Testing , 2003, The Journal of Supercomputing.

[25]  P. M. Melliar-Smith,et al.  Synchronizing clocks in the presence of faults , 1985, JACM.

[26]  Robert M. Hierons,et al.  Using Time to Add Order to Distributed Testing , 2012, FM.

[27]  Gregor von Bochmann,et al.  Generating Synchronizable Test Sequences Based on Finite State Machine with Distributed Ports , 1993, Protocol Test Systems.

[28]  Thierry Jéron,et al.  Towards Automatic Distribution of Testers for Distributed Conformance Testing , 1998, FORTE.

[29]  Marie-Claude Gaudel,et al.  Testing Can Be Formal, Too , 1995, TAPSOFT.

[30]  Marius Mikucionis,et al.  Formal Methods and Testing , 2008 .

[31]  Hasan Ural,et al.  Constructing checking sequences for distributed testing , 2005, Formal Aspects of Computing.

[32]  Gregor von Bochmann,et al.  Synchronization and Specification Issues in Protocol Testing , 1984, IEEE Trans. Commun..

[33]  Kuo-Chung Tai,et al.  Synchronizable Test Sequences of Finite State Machines , 1995, Comput. Networks.

[34]  Colin J. Fidge A Limitation of Vector Timestamps for Reconstructing Distributed Computations , 1998, Inf. Process. Lett..

[35]  Rocco De Nicola,et al.  Asynchronous Observations of Processes , 1998, FoSSaCS.

[36]  Robert M. Hierons,et al.  Implementation Relations for the Distributed Test Architecture , 2008, TestCom/FATES.

[37]  Rance Cleaveland,et al.  Using formal specifications to support testing , 2009, CSUR.

[38]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[39]  Jan Tretmans,et al.  Conformance Testing with Labelled Transition Systems: Implementation Relations and Test Generation , 1996, Comput. Networks ISDN Syst..

[40]  Edward F. Moore,et al.  Gedanken-Experiments on Sequential Machines , 1956 .