Complete model-based equivalence class testing for nondeterministic systems

The main objective of this article is to present a complete finite black-box testing theory for non-deterministic Kripke structures with possibly infinite input domains, but finite domains for internal state variables and outputs. To this end, an abstraction from Kripke structures of this sub-domain to finite state machines is developed. It is shown that every complete black-box testing theory for (deterministic or nondeterministic) finite state machines in the range of this abstraction induces a complete black-box input equivalence class partition testing (IECPT) theory for the Kripke structures under consideration. Additionally, it is shown that each of these IECPT theories can be combined with random testing, such that a random value is selected from an input equivalence class, whenever a representative from this class is required in a test step. Experiments have shown that this combination increases the test strength of equivalence class tests for systems under test (SUT) outside the fault domain, while we show here that this randomisation preserves the completeness property for SUT inside the domain. The investigations lead to several complete IECPT strategies which, to our best knowledge, were not known before for this sub-domain of Kripke structures. The elaboration and presentation of results is performed on a semantic level, so that the testing theories under consideration can be applied to models presented in any concrete formalism, whose behaviour is reflected by a member of our semantic category.

[1]  Jan Peleska,et al.  A Novel Approach to HW/SW Integration Testing of Route-Based Interlocking System Controllers , 2016, RSSRail.

[2]  Jan Peleska,et al.  Experimental evaluation of a novel equivalence class partition testing strategy , 2017, Software & Systems Modeling.

[3]  Matthew Hennessy,et al.  Algebraic theory of processes , 1988, MIT Press series in the foundations of computing.

[4]  C. A. R. Hoare,et al.  Unifying theories of programming , 1998, RelMiCS.

[5]  A. W. Roscoe Understanding Concurrent Systems , 2010, Texts in Computer Science.

[6]  Thomas Santen,et al.  Automating Test Case Generation from Z Specifications with Isabelle , 1997, ZUM.

[7]  Alexandre Petrenko,et al.  Test Selection Based on Communicating Nondeterministic Finite-State Machines Using a Generalized WP-Method , 1994, IEEE Trans. Software Eng..

[8]  G. Bochmann,et al.  Testing deterministic implementations from nondeterministic FSM specifications , 1996 .

[9]  Joseph A. Goguen,et al.  Institutions: abstract model theory for specification and programming , 1992, JACM.

[10]  Myra B. Cohen,et al.  An orchestrated survey of methodologies for automated software test case generation , 2013, J. Syst. Softw..

[11]  Margus Veanes,et al.  Generating finite state machines from abstract state machines , 2002, ISSTA '02.

[12]  Alexandre Petrenko,et al.  Generalizing the DS-Methods for Testing Non-Deterministic FSMs , 2015, Comput. J..

[13]  Jan Peleska,et al.  Formal methods and the development of dependable systems , 1996, Bericht / Christian-Albrechts-Universität Kiel / Institut für Informatik und Praktische Mathematik.

[14]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[15]  Grigore Rosu,et al.  Institution Morphisms , 2013, Formal Aspects of Computing.

[16]  R. Diaconescu Institution-independent model theory , 2008 .

[17]  Markus Roggenbach,et al.  Structured CSP - A Process Algebra as an Institution , 2006, WADT.

[18]  Marie-Claude Gaudel,et al.  Testing Can Be Formal, Too , 1995, TAPSOFT.

[19]  Jan Tretmans,et al.  Conformance Testing with Labelled Transition Systems: Implementation Relations and Test Generation , 1996, Comput. Networks ISDN Syst..

[20]  Nina Yevtushenko,et al.  Adaptive Testing of Deterministic Implementations Specified by Nondeterministic FSMs , 2011, ICTSS.

[21]  Jan Peleska,et al.  Industrial-Strength Model-Based Testing - State of the Art and Current Challenges , 2013, MBT.

[22]  Robert M. Hierons,et al.  Testing from a nondeterministic finite state machine using adaptive state counting , 2004, IEEE Transactions on Computers.

[23]  Ana Cavalcanti,et al.  Testing for refinement in Circus , 2011, Acta Informatica.

[24]  Jan Tretmans,et al.  Test Generation Based on Symbolic Specifications , 2004, FATES.

[25]  Jan Peleska,et al.  Test automation of safety-critical reactive systems , 1997 .

[26]  Nina Yevtushenko,et al.  Fault Models for Testing in Context , 1996, FORTE.

[27]  Anne Elisabeth Haxthausen,et al.  Formal Modeling and Verification of Interlocking Systems Featuring Sequential Release , 2014, FTSCS.

[28]  Insup Lee,et al.  A Temporal Logic Based Theory of Test Coverage and Generation , 2002, TACAS.

[29]  Jan Peleska,et al.  Automated Test Case Generation with SMT-Solving and Abstract Interpretation , 2011, NASA Formal Methods.

[30]  Jeremy Dick,et al.  Automating the Generation and Sequencing of Test Cases from Model-Based Specifications , 1993, FME.

[31]  Jan Peleska,et al.  Complete model-based equivalence class testing , 2014, International Journal on Software Tools for Technology Transfer.

[32]  Robert M. Hierons,et al.  Generating Feasible Transition Paths for Testing from an Extended Finite State Machine (EFSM) , 2009, 2009 International Conference on Software Testing Verification and Validation.

[33]  Edmund M. Clarke,et al.  Model Checking , 1999, Handbook of Automated Reasoning.

[34]  M. P. Vasilevskii Failure diagnosis of automata , 1973 .

[35]  Jim Woodcock,et al.  CSP and Kripke Structures , 2015, ICTAC.

[36]  Ferhat Khendek,et al.  Test Selection Based on Finite State Models , 1991, IEEE Trans. Software Eng..

[37]  Frits W. Vaandrager,et al.  Testing timed automata , 1997, Theor. Comput. Sci..

[38]  Nina Yevtushenko,et al.  Adaptive Testing of Nondeterministic Systems with FSM , 2014, 2014 IEEE 15th International Symposium on High-Assurance Systems Engineering.

[39]  Anne Elisabeth Haxthausen,et al.  Complete Model-Based Equivalence Class Testing for the ETCS Ceiling Speed Monitor , 2014, ICFEM.