A Genetic Algorithm Approach to Focused Software Usage Testing

Because software system testing typically consists of only a very small sample from the set of possible scenarios of system use, it can be difficult or impossible to generalize the test results from a limited amount of testing based on high-level usage models. It can also be very difficult to determine the nature and location of the errors that caused any failures experienced during system testing (and therefore very difficult for the developers to find and fix these errors). To address these issues, this paper presents a Genetic Algorithm (GA) approach to focused software usage testing. Based on the results of macro-level software system testing, a GA is used to select additional test cases to focus on the behavior around the initial test cases to assist in identifying and characterizing the types of test cases that induce system failures (if any) and the types of test cases that do not induce system failures. Whether or not any failures are experienced, this GA approach supports increased test automation and provides increased evidence to support reasoning about the overall quality of the software. When failures are experienced, the approach can improve the efficiency of debugging activities by providing information about similar, but different, test cases that reveal faults in the software and about the input values that triggered the faults to induce failures.

[1]  Lance D. Chambers.  The Practical Handbook of Genetic Algorithms: Applications, Second Edition , 2000 .

[2]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[3]  Andreas Zeller.  Automated Debugging: Are We Close , 2001, Computer.

[4]  David E. Goldberg,et al.  Real-coded Genetic Algorithms, Virtual Alphabets, and Blocking , 1991, Complex Syst..

[5]  Hoang Pham.  Software Reliability , 1999 .

[6]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[7]  Michael R. Lyu,et al.  What is software reliability? , 1994, Proceedings of COMPASS'94 - 1994 IEEE 9th Annual Conference on Computer Assurance.

[8]  Chang Wook Ahn,et al.  On the practical genetic algorithms , 2005, GECCO '05.

[9]  K.A. De Jong,et al.  Adaptive testing of controllers for autonomous vehicles , 1992, Proceedings of the 1992 Symposium on Autonomous Underwater Vehicle Technology.

[10]  David Leon,et al.  Pursuing failure: the distribution of program failures in a profile space , 2001, ESEC/FSE-9.

[11]  Kalyanmoy Deb,et al.  Multi-objective Genetic Algorithms: Problem Difficulties and Construction of Test Problems , 1999, Evolutionary Computation.

[12]  Charles L. Karr,et al.  Industrial Applications of Genetic Algorithms , 1998 .

[13]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[14]  David Coley,et al.  Introduction to Genetic Algorithms for Scientists and Engineers , 1999 .

[15]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[16]  Peter J. Fleming,et al.  An Overview of Evolutionary Algorithms in Multiobjective Optimization , 1995, Evolutionary Computation.

[17]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[18]  Testing Software Based Systems : The Final Frontier Continued from page 1 The Real World of Software Development-A Sobering Perspective , .

[19]  Samir W. Mahfoud.  Niching methods for genetic algorithms , 1996 .

[20]  Jeffrey Horn,et al.  The nature of niching: genetic algorithms and the evolution of optimal, cooperative populations , 1997 .