Runtime analysis of (1+l) EA on computing unique input output sequences

Computing unique input output (UIO) sequences is a fundamental and hard problem in conformance testing of finite state machines (FSM). Previous experimental research has shown that evolutionary algorithms (EAs) can be applied successfully to find UIOs on some instances. However, before EAs can be recommended as a practical technique for computing UIOs, it is necessary to better understand the potential and limitations of these algorithms on this problem. In particular, more research is needed in determining for what instances of the problem EAs are feasible. This paper presents a rigorous runtime analysis of the (1+1) EA on three classes of instances of this problem. First, it is shown that there are instances where the EA is efficient, while random testing fails completely. Secondly, an instance class that is difficult for both random testing and the EA is presented. Finally, a parametrised instance class with tunable difficulty is presented. Together, these results provide a first theoretical characterisation of the potential and limitations of the (1+1) EA on the problem of computing UIOs.

[1]  Per Kristian Lehre,et al.  Theoretical Runtime Analyses of Search Algorithms on the Test Data Generation for the Triangle Classification Problem , 2008, 2008 IEEE International Conference on Software Testing Verification and Validation Workshop.

[2]  Xin Yao,et al.  Time complexity of evolutionary algorithms for combinatorial optimization: A decade of results , 2007, Int. J. Autom. Comput..

[3]  Ingo Wegener,et al.  Real royal road functions--where crossover provably is essential , 2001, Discret. Appl. Math..

[4]  Mark Harman,et al.  Computing Unique Input/Output Sequences Using Genetic Algorithms , 2003, FATES.

[5]  Karnig Derderian,et al.  Automated test sequence generation for finite state machines using genetic algorithms , 2006 .

[6]  Mark Harman,et al.  Generating feasible input sequences for extended finite state machines (EFSMs) using genetic algorithms , 2005, GECCO '05.

[7]  Frank Neumann,et al.  Rigorous analyses of simple diversity mechanisms , 2007, GECCO '07.

[8]  Ingo Wegener,et al.  Real royal road functions for constant population size , 2003, Theor. Comput. Sci..

[9]  Thomas Jansen,et al.  UNIVERSITY OF DORTMUND REIHE COMPUTATIONAL INTELLIGENCE COLLABORATIVE RESEARCH CENTER 531 Design and Management of Complex Technical Processes and Systems by means of Computational Intelligence Methods Upper and Lower Bounds for Randomized Search Heuristics in Black-Box Optimization , 2004 .

[10]  Mark Harman,et al.  Automated Unique Input Output Sequence Generation for Conformance Testing of FSMs , 2006, Comput. J..

[11]  Mark Harman,et al.  A theoretical & empirical analysis of evolutionary testing and hill climbing for structural test data generation , 2007, ISSTA '07.

[12]  Mark Harman,et al.  Improving test quality using robust unique input/output circuit sequences (UIOCs) , 2006, Inf. Softw. Technol..

[13]  Thomas Jansen,et al.  On the analysis of the (1+1) evolutionary algorithm , 2002, Theor. Comput. Sci..

[14]  Rajeev Motwani,et al.  Randomized Algorithms , 1995, SIGA.

[15]  Alexandre Petrenko,et al.  Protocol testing: review of methods and relevance for software testing , 1994, ISSTA '94.

[16]  Schloss Birlinghoven,et al.  How Genetic Algorithms Really Work I.mutation and Hillclimbing , 2022 .

[17]  Mark Harman,et al.  Reformulating software engineering as a search problem , 2003 .

[18]  Thomas Jansen,et al.  On the brittleness of evolutionary algorithms , 2007, FOGA'07.

[19]  Ingo Wegener,et al.  Fitness Landscapes Based on Sorting and Shortest Paths Problems , 2002, PPSN.

[20]  David Lee,et al.  Testing Finite-State Machines: State Identification and Verification , 1994, IEEE Trans. Computers.

[21]  Benjamin Doerr,et al.  Faster Evolutionary Algorithms by Superior Graph Representation , 2007, 2007 IEEE Symposium on Foundations of Computational Intelligence.

[22]  Xin Yao,et al.  A study of drift analysis for estimating computation time of evolutionary algorithms , 2004, Natural Computing.

[23]  Heinz Mühlenbein,et al.  How Genetic Algorithms Really Work: Mutation and Hillclimbing , 1992, PPSN.

[24]  Günter Rudolph,et al.  Finite Markov Chain Results in Evolutionary Computation: A Tour d'Horizon , 1998, Fundam. Informaticae.

[25]  R. K. Shyamasundar,et al.  Introduction to algorithms , 1996 .

[26]  Oliver Giel,et al.  Zur Analyse von randomisierten Suchheuristiken und Online-Heuristiken , 2005 .

[27]  Deepinder P. Sidhu,et al.  Formal Methods for Protocol Testing: A Detailed Study , 1989, IEEE Trans. Software Eng..

[28]  Mark Harman,et al.  Constructing multiple unique input/output sequences using metaheuristic optimisation techniques , 2005, IEE Proc. Softw..

[29]  Stefan Droste,et al.  A rigorous analysis of the compact genetic algorithm for linear functions , 2006, Natural Computing.

[30]  Mark Harman,et al.  Heuristics for fault diagnosis when testing from finite state machines , 2007, Softw. Test. Verification Reliab..

[31]  Mark Harman,et al.  The Current State and Future of Search Based Software Engineering , 2007, Future of Software Engineering (FOSE '07).

[32]  Ingo Wegener,et al.  Randomized local search, evolutionary algorithms, and the minimum spanning tree problem , 2004, Theor. Comput. Sci..

[33]  Qiang Guo,et al.  Improving fault coverage and minimising the cost of fault identification when testing from finite state machines , 2006 .

[34]  David Lee,et al.  Principles and methods of testing finite state machines-a survey , 1996, Proc. IEEE.

[35]  Carsten Witt,et al.  Population size versus runtime of a simple evolutionary algorithm , 2008, Theor. Comput. Sci..

[36]  Per Kristian Lehre,et al.  On the effect of populations in evolutionary multi-objective optimization , 2006, GECCO.

[37]  Pietro Simone Oliveto,et al.  Design and Management of Complex Technical Processes and Systems by Means of Computational Intelligence Methods Theoretical Analysis of Diversity Mechanisms for Global Exploration Theoretical Analysis of Diversity Mechanisms for Global Exploration , 2022 .

[38]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..