Model Based Testing with Labelled Transition Systems

Model based testing is one of the promising technologies to meet the challenges imposed on software testing. In model based testing an implementation under test is tested for compliance with a model that describes the required behaviour of the implementation. This tutorial chapter describes a model based testing theory where models are expressed as labelled transition systems, and compliance is defined with the 'ioco' implementation relation. The ioco-testing theory, on the one hand, provides a sound and well-defined foundation for labelled transition system testing, having its roots in the theoretical area of testing equivalences and refusal testing. On the other hand, it has proved to be a practical basis for several model based test generation tools and applications. Definitions, underlying assumptions, an algorithm, properties, and several examples of the ioco-testing theory are discussed, involving specifications, implementations, tests, the ioco implementation relation and some of its variants, a test generation algorithm, and the soundness and exhaustiveness of this algorithm.

[1]  Rocco De Nicola,et al.  Testing Equivalences for Processes , 1984, Theor. Comput. Sci..

[2]  Bertrand Jeannet,et al.  Symbolic Test Selection Based on Approximate Analysis , 2005, TACAS.

[3]  Tommaso Bolognesi,et al.  Tableau methods to describe strong bisimilarity on LOTOS processes involving pure interleaving and enabling , 1994, FORTE.

[4]  Marie-Claude Gaudel,et al.  Testing Can Be Formal, Too , 1995, TAPSOFT.

[5]  Giuseppe Scollo,et al.  Lotos specifications, their implementations and their tests , 1995 .

[6]  Lex Heerink,et al.  Ins and Outs in Refusal Testing , 1998 .

[7]  Margus Veanes,et al.  Model-Based Testing of Object-Oriented Reactive Systems with Spec Explorer , 2008, Formal Methods and Testing.

[8]  Marie-Claude Gaudel,et al.  Software testing based on formal specifications: a theory and a tool , 1991, Softw. Eng. J..

[9]  Ed Brinksma,et al.  A Test Generation Framework for quiescent Real-Time Systems , 2004, FATES.

[10]  Jan Tretmans,et al.  Testing Concurrent Systems: A Formal Approach , 1999, CONCUR.

[11]  Olivier Charles,et al.  Relating Conformance Test Coverage to Formal Specifications , 1996, FORTE.

[12]  Alexandre Petrenko,et al.  Fault Model-Driven Test Derivation from Finite State Models: Annotated Bibliography , 2000, MOVEP.

[13]  Nina Yevtushenko,et al.  Testing Transition Systems with Input and Output Testers , 2003, TestCom.

[14]  Jan Tretmans,et al.  Test Generation Based on Symbolic Specifications , 2004, FATES.

[15]  Thierry Jéron,et al.  A tool for the automatic synthesis of conformance test cases for non-deterministic reactive systems , 2005 .

[16]  Son T. Vuong,et al.  Sensitivity analysis of the metric based test selection , 1997 .

[17]  Rom Langerak.  A Testing Theory for LOTOS using Deadlock Detection , 1989, PSTV.

[18]  Jan Tretmans,et al.  Test Generation with Inputs, Outputs and Repetitive Quiescence , 1996, Softw. Concepts Tools.

[19]  Marinus J. Plasmeijer,et al.  Gast: Generic Automated Software Testing , 2002, IFL.

[20]  Jan Tretmans,et al.  Testing Transition Systems: An Annotated Bibliography , 2000, MOVEP.

[21]  Kenneth J. Turner,et al.  Protocol-Inspired Hardware Testing , 1999, IWTCS.

[22]  K. Larsen,et al.  Online Testing of Real-time Systems Using Uppaal , 2004, FATES.

[23]  Robin Milner,et al.  Communication and concurrency , 1989, PHI Series in computer science.

[24]  Jan Tretmans,et al.  TorX: Automated Model-Based Testing , 2003 .

[25]  Nicolae Goga,et al.  Test Selection, Trace Distance and Heuristics , 2002, TestCom.

[26]  Roberto Segala.  Quiescence, Fairness, Testing, and the Notion of Implementation (Extended Abstract) , 1993, CONCUR.

[27]  Iso. Lotos,et al.  A Formal Description Technique Based on the Temporal Ordering of Observational Behaviour , 1985 .

[28]  Alan Hartman,et al.  The AGEDIS tools for model based testing , 2004, ISSTA '04.

[29]  Margus Veanes,et al.  Testing Concurrent Object-Oriented Systems with Spec Explorer , 2005, FM.

[30]  Thierry Heuillard,et al.  AGEDIS Case Studies: Model-Based Testing in Industry , 2003 .

[31]  R. G. de Vries.  Towards Formal Test Purposes , 2001 .

[32]  Nancy A. Lynch,et al.  An introduction to input/output automata , 1989 .

[33]  Arend Rensink,et al.  Compositional Testing with ioco , 2003, FATES.

[34]  Marc Phalippou.  Relations d'implantation et hypothèses de test sur des automates à entrées et sorties , 1994 .

[35]  Arend Rensink,et al.  Action Refinement in Conformance Testing , 2005, TestCom.

[36]  Roland Groz,et al.  Eight Years of Experience in Test Generation from FDTs using TVEDA , 1997, FORTE.

[37]  Stavros Tripakis,et al.  Conformance testing for real-time systems , 2004, SPIN.

[38]  Axel Belinfante,et al.  Automated Testing in Practice: The Highway Tolling System , 2002, TestCom.

[39]  Roberto Segala,et al.  Quiescence, Fairness, Testing, and the Notion of Implementation , 1997, Inf. Comput..

[40]  Duncan Clarke,et al.  Automated Test and Oracle Generation for Smart-Card Applications , 2001, E-smart.

[41]  Alexandre Petrenko,et al.  On Testing Partially Specified IOTS through Lossless Queues , 2004, TestCom.

[42]  Frits W. Vaandrager,et al.  On the relationship between process algebra and input/output automata , 1991, [1991] Proceedings Sixth Annual IEEE Symposium on Logic in Computer Science.

[43]  Ed Brinksma,et al.  On the Coverage of Partial Validations , 1993, AMAST.

[44]  Rocco De Nicola,et al.  Extensional equivalences for transition systems , 1987, Acta Informatica.

[45]  Iain Phillips,et al.  Refusal Testing , 1986, Theoretical Computer Science.

[46]  Jan Tretmans,et al.  A Formal Approach to Conformance Testing , 1993, Protocol Test Systems.