Relating Alternating Relations for Conformance and Refinement ? Technical Report

Various relations have been defined to express refinement and conformance for state-transition systems with inputs and outputs, such as ioco and uioco in the area of model-based testing, and alternating simulation and alternating-trace containment originating from game theory and formal verification. Several papers have compared these independently developed relations, but these comparisons make assumptions (e.g., input-enabledness), pose restrictions (e.g., determinism – then they all coincide), use different models (e.g., interface automata and Kripke structures), or do not deal with the concept of quiescence. In this paper, we present the integration of the ioco/uioco theory of model-based testing and the theory of alternating refinements, within the domain of non-deterministic, non-input-enabled interface automata. A standing conjecture is that ioco and alternating-trace containment coincide. Our main result is that this conjecture does not hold, but that uioco coincides with a variant of alternating-trace containment, for image finite interface automata and with explicit treatment of quiescence. From the comparison between ioco theory and alternating refinements, we conclude that ioco and the original relation of alternating-trace containment are too strong for realistic black-box scenarios. We present a refinement relation which can express both uioco and refinement in game theory, while being simpler and having a clearer observational interpretation.

[1]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[2]  Samson Abramsky,et al.  Observation Equivalence as a Testing Equivalence , 1987, Theor. Comput. Sci..

[3]  Nancy A. Lynch,et al.  An introduction to input/output automata , 1989 .

[4]  Frits W. Vaandrager,et al.  On the relationship between process algebra and input/output automata , 1991, [1991] Proceedings Sixth Annual IEEE Symposium on Logic in Computer Science.

[5]  Da-Wei Wang,et al.  Games I/O Automata Play (Extended Abstract) , 1992, CONCUR.

[6]  Marie-Claude Gaudel,et al.  Testing Can Be Formal, Too , 1995, TAPSOFT.

[7]  Jan Tretmans,et al.  Test Generation with Inputs, Outputs and Repetitive Quiescence , 1996, Softw. Concepts Tools.

[8]  Lex Heerink,et al.  Refusal Testing for Classes of Transition Systems with Inputs and Outputs , 1997, FORTE.

[9]  Thomas A. Henzinger,et al.  Alternating Refinement Relations , 1998, CONCUR.

[10]  Nancy A. Lynch,et al.  Liveness in Timed and Untimed Systems , 1998, Inf. Comput..

[11]  Thomas A. Henzinger,et al.  Interface automata , 2001, ESEC/FSE-9.

[12]  Thomas A. Henzinger,et al.  Synchronous and Bidirectional Component Interfaces , 2002, CAV.

[13]  Arend Rensink,et al.  Compositional Testing with ioco , 2003, FATES.

[14]  Jan Tretmans,et al.  Model-Based Testing of Environmental Conformance of Components , 2006, FMCO.

[15]  R. V. Glabbeek The Linear Time-Branching Time Spectrum I The Semantics of Concrete , Sequential ProcessesR , 2007 .

[16]  Jan Tretmans,et al.  Model Based Testing with Labelled Transition Systems , 2008, Formal Methods and Testing.

[17]  Frits W. Vaandrager,et al.  Learning I/O Automata , 2010, CONCUR.

[18]  Nikolaj Bjørner,et al.  Alternating Simulation and IOCO , 2010, ICTSS.

[19]  Jan Tretmans,et al.  Towards quality of model-based testing in the ioco framework , 2013, JAMAICA 2013.

[20]  Bengt Jonsson,et al.  An algebraic theory of interface automata , 2014, Theor. Comput. Sci..

[21]  Mariëlle Stoelinga,et al.  Tester versus Bug: A Generic Framework for Model-Based Testing via Games , 2018, GandALF.

[22]  Jan Tretmans,et al.  Matching implementations to specifications: the corner cases of ioco , 2019, SAC.