Scatter Search in software testing, comparison and collaboration with Estimation of Distribution Algorithms

Abstract One of the main tasks software testing includes is the generation of the test cases to be used during the test. Due to its expensive cost, the automatization of this task has become one of the key issues in the area. The field of Evolutionary Testing deals with this problem by means of metaheuristic search techniques. An Evolutionary Testing based approach to the automatic generation of test inputs is presented. The approach developed involves different possibilities of the usage of two heuristic optimization methods, namely, Scatter Search and Estimation of Distribution Algorithms. The possibilities comprise pure Scatter Search options and Scatter Search—Estimation of Distribution Algorithm collaborations. Several experiments were conducted in order to evaluate and compare the approaches presented with those in the literature. The analysis of the experimental results raises interesting conclusions, showing these alternatives as a promising option to tackle this problem.

[1]  Jin-Cherng Lin,et al.  Automatic test data generation for path testing using GAs , 2001, Inf. Sci..

[2]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[3]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[4]  Boris Beizer,et al.  Software Testing Techniques , 1983 .

[5]  Joachim Wegener,et al.  Testing the temporal behavior of real-time software modules using extended evolutionary algorithms , 1999 .

[6]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[7]  Roy P. Pargas,et al.  Test‐data generation using genetic algorithms , 1999, Softw. Test. Verification Reliab..

[8]  Rafael Martí,et al.  Scatter Search: Diseño Básico y Estrategias avanzadas , 2002, Inteligencia Artif..

[9]  Bryan F. Jones,et al.  Automatic structural testing using genetic algorithms , 1996, Softw. Eng. J..

[10]  D. R. Lick,et al.  Graph theory with applications to algorithms and computer science , 1985 .

[11]  Jim Smith,et al.  Evolving Software Test Data - GA's learn Self Expression , 1996, Evolutionary Computing, AISB Workshop.

[12]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[13]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[14]  Simeon C. Ntafos,et al.  An Evaluation of Random Testing , 1984, IEEE Transactions on Software Engineering.

[15]  P. Larra,et al.  Feature Subset Selection by Bayesian Networks Based Optimization Abstract|a New Method for Feature Subset Selection in Machine Learning, Fss-ebna , 1999 .

[16]  Fred W. Glover,et al.  An Experimental Evaluation of a Scatter Search for the Linear Ordering Problem , 2001, J. Glob. Optim..

[17]  Gregory F. Cooper,et al.  A Bayesian Method for the Induction of Probabilistic Networks from Data , 1992 .

[18]  Mark Harman,et al.  Improving Evolutionary Testing By Flag Removal , 2002, GECCO.

[19]  S. Baluja,et al.  Combining Multiple Optimization Runs with Optimal Dependency Trees , 1997 .

[20]  A. Jefferson Offutt,et al.  Experimental results from an automatic test case generator , 1993, TSEM.

[21]  F. Glover,et al.  Fundamentals of Scatter Search and Path Relinking , 2000 .

[22]  Gary McGraw,et al.  Generating Software Test Data by Evolution , 2001, IEEE Trans. Software Eng..

[23]  John A. Clark,et al.  An automated framework for structural test-data generation , 1998, Proceedings 13th IEEE International Conference on Automated Software Engineering (Cat. No.98EX239).

[24]  Paul A. Viola,et al.  MIMIC: Finding Optima by Estimating Probability Densities , 1996, NIPS.

[25]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[26]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[27]  Harmen-Hinrich Sthamer,et al.  The automatic generation of software test data using genetic algorithms , 1995 .

[28]  André Baresel,et al.  Evolutionary Testing of Flag Conditions , 2003, GECCO.

[29]  Boris Beizer,et al.  Black Box Testing: Techniques for Functional Testing of Software and Systems , 1996, IEEE Software.

[30]  R. Sagarna,et al.  Variable search space for software testing , 2003, International Conference on Neural Networks and Signal Processing, 2003. Proceedings of the 2003.

[31]  Bogdan Korel,et al.  Automated Software Test Data Generation , 1990, IEEE Trans. Software Eng..

[32]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[33]  Heinz Mühlenbein,et al.  The Equation for Response to Selection and Its Use for Prediction , 1997, Evolutionary Computation.

[34]  Pedro Larrañaga,et al.  Feature Subset Selection by Bayesian network-based optimization , 2000, Artif. Intell..

[35]  BayesiannetworksPedro,et al.  Combinatorial optimization by learning and simulation of , 2000 .

[36]  N. E. Fenton.  The structral complexity of flowgraphs , 1985 .

[37]  Leonardo Bottaci,et al.  Predicate Expression Cost Functions to Guide Evolutionary Search for Test Data , 2003, GECCO.

[38]  Rafael Martí,et al.  Experimental Testing of Advanced Scatter Search Designs for Global Optimization of Multimodal Functions , 2005, J. Glob. Optim..

[39]  Simon Burton.  Automated Testing From Z Specifications , 2000 .

[40]  Jose A. Lozano.  Scatter Search and Estimation of Distribution Algorithms in Software Testing: Competition and Cooperation , 2003 .

[41]  James Miller,et al.  Genetic Algorithms and the Automatic Generation ofTest , 1995 .

[42]  Yoichi Hayashi,et al.  Neural expert system using fuzzy teaching input and its application to medical diagnosis , 1994 .

[43]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[44]  José Antonio Lozano,et al.  On The Performance Of Estimation Of Distribution Algorithms Applied To Software Testing , 2005, Appl. Artif. Intell..

[45]  Joachim Wegener,et al.  Evolutionary test environment for automatic structural testing , 2001, Inf. Softw. Technol..

[46]  Vicente Campos,et al.  Scatter Search vs. Genetic Algorithms , 2002 .

[47]  Eugenia Díaz,et al.  Automated software testing using a metaheuristic technique based on Tabu search , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..