Unsupervised test model reconstruction from conformance test logs

The high manpower cost of test model generation is a big obstacle for model based testing. In this paper, we propose a method that decreases this cost for the case of performance testing of telecommunication systems. The test model which is typically a deterministic finite state machine is partially derived from execution logs of black box system tests by means of statistical methods and then manually minimized. A sequential pattern mining algorithm is used for identifying frequently observed event sequences, from which it is possible to identify partitions of states of the machine and set up functional dependencies among them. Our method in contrast to other methods is passive and unsupervised it does neither require access to the black box system to be learnt, nor the input sequences to be finite. However, the equivalence of some states cannot be proven, therefore the model has to be finalized manually. We show experimental results on random incompletely specified FSMs and sets of random walk traces to evaluate the efficiency of our method.

[1]  Charles P. Pfleeger,et al.  State Reduction in Incompletely Specified Finite-State Machines , 1973, IEEE Transactions on Computers.

[2]  F. Joel Ferguson,et al.  On state reduction of incompletely specified finite state machines , 2007, Comput. Electr. Eng..

[3]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[4]  Ramakrishnan Srikant,et al.  Mining Sequential Patterns: Generalizations and Performance Improvements , 1996, EDBT.

[5]  Ramakrishnan Srikant,et al.  Fast Algorithms for Mining Association Rules in Large Databases , 1994, VLDB.

[6]  David Lee,et al.  Principles and methods of testing finite state machines-a survey , 1996, Proc. IEEE.

[7]  Hardi Hungar,et al.  Model Generation by Moderated Regular Extrapolation , 2002, FASE.

[8]  Howard Barringer,et al.  Proof Rules for Automated Compositional Verification through Learning , 2003 .

[9]  Keqin Li,et al.  Integration Testing of Components Guided by Incremental State Machine Learning , 2006, Testing: Academic & Industrial Conference - Practice And Research Techniques (TAIC PART'06).

[10]  José L. Balcázar,et al.  Algorithms for Learning Finite Automata from Queries: A Unified View , 1997, Advances in Algorithms, Languages, and Complexity.

[11]  Keqin Li,et al.  Learning Parameterized State Machine Model for Integration Testing , 2007, 31st Annual International Computer Software and Applications Conference (COMPSAC 2007).

[12]  Gend Lal Prajapati,et al.  Advances in Learning Formal Languages , 2011 .

[13]  Hardi Hungar,et al.  Domain-Specific Optimization in Automata Learning , 2003, CAV.

[14]  Bernhard Steffen,et al.  Automata Learning with Automated Alphabet Abstraction Refinement , 2011, VMCAI.

[15]  Ronald L. Rivest,et al.  Inference of finite automata using homing sequences , 1989, STOC '89.