Effective algorithms for constructing minimum cost adaptive distinguishing sequences

Context: Given a Finite State Machine (FSM), a checking sequence is a test sequence that determines whether the system under test is correct as long as certain standard assumptions hold. Many checking sequence generation methods use an adaptive distinguishing sequence (ADS), which is an experiment that distinguishes the states of the specification machine. Furthermore, it has been shown that the use of shorter ADSs yields shorter checking sequences. It is also known, on the other hand, that constructing a minimum cost ADS is an NP-hard problem and it is NP-hard to approximate. This motivates studying and investigating effective ADS construction methods.Objective: The main objective of this paper is to suggest new methods that can compute compact ADSs to be used in the construction of checking sequences.Method: We briefly present the existing ADS construction algorithms. We then propose generalizations of these approaches with a set of heuristics. We also conduct experiments to compare the size of the resultant ADSs and the length of the checking sequences constructed using these ADSs.Results: The results indicate that when the ADSs are constructed with the proposed methods, the length of the checking sequences may reduce up to 54% (40% on the average).Conclusions: In this paper, we present the state of the art ADS construction methods for FSMs and we propose generalizations of these methods. We show that our methods are effective in terms of computation time and ADS quality.

[1]  May Haydar,et al.  Formal Verification of Web Applications Modeled by Communicating Automata , 2004, FORTE.

[2]  Uraz Cengiz Türker,et al.  Lookahead-Based Approaches for Minimizing Adaptive Distinguishing Sequences , 2014, ICTSS.

[3]  Robert M. Hierons,et al.  Distinguishing Sequences for Partially Specified FSMs , 2014, NASA Formal Methods.

[4]  Hasan Ural,et al.  Optimal length test sequence generation using distinguishing sequences , 1993, TNET.

[5]  Robert M. Hierons,et al.  Optimizing the length of checking sequences , 2006, IEEE Transactions on Computers.

[6]  Nina Yevtushenko,et al.  Some classes of finite state machines with polynomial length of distinguishing test cases , 2016, SAC.

[7]  Xiaolin Wu,et al.  On Minimizing the Lengths of Checking Sequences , 1997, IEEE Trans. Computers.

[8]  Jessica Chen,et al.  Exploring alternatives for transition verification , 2009, J. Syst. Softw..

[9]  I. K. Rystsov Proof of an achievable bound on the length of a conditional diagnostic experiment for a finite automaton , 1976, Cybernetics.

[10]  Hasan Ural,et al.  Reducing the Lengths of Checking Sequences by Overlapping , 2006, TestCom.

[11]  Hasan Ural,et al.  Lower bounds on lengths of checking sequences , 2009, Formal Aspects of Computing.

[12]  Steven H. Low,et al.  Probabilistic conformance testing of protocols with unobservable transitions , 1993, 1993 International Conference on Network Protocols.

[13]  David Harel,et al.  Modeling Reactive Systems With Statecharts : The Statemate Approach , 1998 .

[14]  Robert M. Hierons,et al.  Checking Sequence Construction Using Adaptive and Preset Distinguishing Sequences , 2009, 2009 Seventh IEEE International Conference on Software Engineering and Formal Methods.

[15]  Krishan K. Sabnani,et al.  Formal methods for generating protocol conformance test sequences , 1990, Proc. IEEE.

[16]  F. C. Hennine Fault detecting experiments for sequential circuits , 1964, SWCT 1964.

[17]  Alexandre Petrenko,et al.  Generating Checking Sequences for Partial Reduced Finite State Machines , 2008, TestCom/FATES.

[18]  David Lee,et al.  Conformance testing of protocols specified as communicating finite state machines-a guided random walk based approach , 1996, IEEE Trans. Commun..

[19]  Güney Gönenç,et al.  A Method for the Design of Fault Detection Experiments , 1970 .

[20]  David Lee,et al.  Principles and methods of testing finite state machines-a survey , 1996, Proc. IEEE.

[21]  Hasan Ural,et al.  Generalizing Redundancy Elimination in Checking Sequences , 2005, ISCIS.

[22]  M. N. Sokolovskii Diagnostic experiments with automata , 1971, Cybernetics.

[23]  Wolfgang Grieskamp,et al.  Model‐based quality assurance of protocol documentation: tools and methodology , 2011, Softw. Test. Verification Reliab..

[24]  W. Cleveland Robust Locally Weighted Regression and Smoothing Scatterplots , 1979 .

[25]  Alexandre Petrenko,et al.  Checking Sequence Generation Using State Distinguishing Subsequences , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[26]  Robert M. Hierons,et al.  Parallel Algorithms for Generating Distinguishing Sequences for Observable Non-deterministic FSMs , 2017, ACM Trans. Softw. Eng. Methodol..

[27]  R. T. Boute Distinguishing Sets for Optimal State Identification in Checking Experiments , 1974, IEEE Transactions on Computers.

[28]  Alexandre Petrenko,et al.  Checking Completeness of Tests for Finite State Machines , 2010, IEEE Transactions on Computers.

[29]  Krishan K. Sabnani,et al.  A Protocol Test Generation Procedure , 1988, Comput. Networks.

[30]  Robert M. Hierons,et al.  Reduced Length Checking Sequences , 2002, IEEE Trans. Computers.

[31]  Robert M. Hierons,et al.  Eliminating Redundant Tests in a Checking Sequence , 2005, TestCom.

[32]  Monika Kapus-Kolar On the global optimization of checking sequences for finite state machine implementations , 2014, Microprocess. Microsystems.

[33]  Nina Yevtushenko,et al.  Adaptive Homing and Distinguishing Experiments for Nondeterministic Finite State Machines , 2013, ICTSS.

[34]  David Lee,et al.  Testing Finite-State Machines: State Identification and Verification , 1994, IEEE Trans. Computers.

[35]  Uraz Cengiz Türker,et al.  Hardness and inapproximability of minimizing adaptive distinguishing sequences , 2014, Formal Methods Syst. Des..

[36]  Monika Kapus-Kolar On "Exploring alternatives for transition verification" , 2012, J. Syst. Softw..

[37]  Deepinder P. Sidhu,et al.  Formal Methods for Protocol Testing: A Detailed Study , 1989, IEEE Trans. Software Eng..

[38]  Robert M. Hierons,et al.  Incomplete Distinguishing Sequences for Finite State Machines , 2015, Comput. J..

[39]  Bruno Legeard,et al.  A taxonomy of model‐based testing approaches , 2012, Softw. Test. Verification Reliab..

[40]  Hüsnü Yenigün,et al.  Using Distinguishing and UIO Sequences Together in a Checking Sequence , 2006, TestCom.

[41]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.