Checking Experiments for Symbolic Input/Output Finite State Machines

After sixty or so years of development the theory of checking experiments for FSM still continues to attract a lot of attention of research community. One of the reasons is that it offers test generation techniques which under well-defined assumptions guarantee complete fault coverage for a given fault model of a specification FSM. Checking experiments have already been extended to remove assumptions that the specification Mealy machine need to be reduced, deterministic, and completely specified, while keeping the input, output and state sets finite. In our recent work, we investigated possibilities of removing the assumption about the finiteness of the input set, introducing the model FSM with symbolic inputs. In this paper, we report on our efforts of further lifting the theory of checking experiments for Mealy machines with symbolic inputs and symbolic outputs. The former are predicates defined over input variables and the latter are output variable valuations computed by assignments on input variables. Both types of variables can have large or even infinite domains. Inclusion of assignments in the model complicates fault detection, as different assignments may produce the same output valuations for some input valuations. We address this issue by using a transition cover enhanced with assignment discriminating predicates specifying symbolic inputs on which the assignments produce different outputs. The enhanced transition cover is then used in checking experiments, which can detect assignment/output faults and more general transition faults under certain assumptions.

[1]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[2]  Gertjan van Noord,et al.  Finite State Transducers with Predicates and Identities , 2001, Grammars.

[3]  G. Bochmann,et al.  Fault Models in Testing , 1991, Protocol Test Systems.

[4]  David Lee,et al.  Testing Finite State Machines: Fault Detection , 1995, J. Comput. Syst. Sci..

[5]  Thierry Jéron,et al.  An Approach to Symbolic Test Generation , 2000, IFM.

[6]  Jan Tretmans,et al.  Test Generation Based on Symbolic Specifications , 2004, FATES.

[7]  Alexandre Petrenko,et al.  Checking Experiments for Finite State Machines with Symbolic Inputs , 2015, ICTSS.

[8]  Nina Yevtushenko,et al.  Fault Models for Testing in Context , 1996, FORTE.

[9]  Paul D. Amer,et al.  Test Case Generation for Protocols Specified in Estelle , 1990, FORTE.

[10]  Andreas Prinz,et al.  The formal semantics of SDL-2000: Status and perspectives , 2003, Comput. Networks.

[11]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[12]  Ana R. Cavalli,et al.  Experimental evaluation of FSM-based testing methods , 2005, Third IEEE International Conference on Software Engineering and Formal Methods (SEFM'05).

[13]  Amnon Naamad,et al.  The STATEMATE semantics of statecharts , 1996, TSEM.

[14]  Paulo César Masiero,et al.  Mutation testing applied to validate specifications based on statecharts , 1999, Proceedings 10th International Symposium on Software Reliability Engineering (Cat. No.PR00443).

[15]  Insup Lee,et al.  A Temporal Logic Based Theory of Test Coverage and Generation , 2002, TACAS.

[16]  A. Tiwari Formal Semantics and Analysis Methods for Simulink Stateflow Models , 2001 .

[17]  Alexandre Petrenko,et al.  A Method and Tool for Test Optimization for Automotive Controllers , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops.

[18]  Alexandre Petrenko,et al.  Synthesis of test experiments in some classes of automata , 1991 .

[19]  Jan Peleska,et al.  Exhaustive Model-Based Equivalence Class Testing , 2013, ICTSS.

[20]  M. P. Vasilevskii Failure diagnosis of automata , 1973 .

[21]  Edward F. Moore,et al.  Gedanken-Experiments on Sequential Machines , 1956 .

[22]  Jan Peleska,et al.  Complete model-based equivalence class testing , 2014, International Journal on Software Tools for Technology Transfer.

[23]  Oded Maler,et al.  Learning Regular Languages over Large Alphabets , 2014, TACAS.

[24]  Robert M. Hierons,et al.  Generating Feasible Transition Paths for Testing from an Extended Finite State Machine (EFSM) , 2009, 2009 International Conference on Software Testing Verification and Validation.

[25]  F. C. Hennie Fault detecting experiments for sequential circuits , 1964, SWCT.

[26]  Nikolaj Bjørner,et al.  Symbolic finite state transducers: algorithms and applications , 2012, POPL '12.

[27]  Ming T. Liu,et al.  Generating test cases for EFSM with given fault models , 1993, IEEE INFOCOM '93 The Conference on Computer Communications, Proceedings.

[28]  Nina Yevtushenko,et al.  Conformance Tests as Checking Experiments for Partial Nondeterministic FSM , 2005, FATES.

[29]  Kwang-Ting Cheng,et al.  Automatic Functional Test Generation Using The Extended Finite State Machine Model , 1993, 30th ACM/IEEE Design Automation Conference.

[30]  Margus Veanes Applications of Symbolic Finite Automata , 2013, CIAA.

[31]  Myra B. Cohen,et al.  An orchestrated survey of methodologies for automated software test case generation , 2013, J. Syst. Softw..

[32]  Teruo Higashino,et al.  Automatic generation of extended UIO sequences for communication protocols in an EFSM model , 1995 .

[33]  Anna Philippou,et al.  Tools and Algorithms for the Construction and Analysis of Systems , 2018, Lecture Notes in Computer Science.

[34]  Nina Yevtushenko,et al.  Test Suite Generation from a FSM with a Given Type of Implementation Errors , 1992, PSTV.

[35]  Ferhat Khendek,et al.  Test Selection Based on Finite State Models , 1991, IEEE Trans. Software Eng..

[36]  Sanford Friedenthal,et al.  A Practical Guide to SysML: The Systems Modeling Language , 2008 .

[38]  Burkhart Wolff,et al.  Symbolic Methods in Testing (Dagstuhl Seminar 13021) , 2013, Dagstuhl Reports.

[39]  Alexandre Petrenko,et al.  Confirming configurations in EFSM testing , 2004, IEEE Transactions on Software Engineering.

[40]  Bruce W. Watson Implementing and using finite automata toolkits , 1996, Nat. Lang. Eng..