Automatic, evolutionary test data generation for dynamic software testing

This paper proposes a dynamic test data generation framework based on genetic algorithms. The framework houses a Program Analyser and a Test Case Generator, which intercommunicate to automatically generate test cases. The Program Analyser extracts statements and variables, isolates code paths and creates control flow graphs. The Test Case Generator utilises two optimisation algorithms, the Batch-Optimistic (BO) and the Close-Up (CU), and produces a near to optimum set of test cases with respect to the edge/condition coverage criterion. The efficacy of the proposed approach is assessed on a number of programs and the empirical results indicate that its performance is significantly better compared to existing dynamic test data generation methods.

[1]  Gary McGraw,et al.  Automated software test data generation for complex programs , 1998, Proceedings 13th IEEE International Conference on Automated Software Engineering (Cat. No.98EX239).

[2]  Yannis Smaragdakis,et al.  Check 'n' crash: combining static checking and testing , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[3]  V. Lakshmi Narasimhan,et al.  ADTEST: A Test Data Generation Suite for Ada Software Systems , 1997, IEEE Trans. Software Eng..

[4]  Donald J. Berndt,et al.  Breeding software test cases for complex systems , 2004, 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the.

[5]  Mary Jean Harrold,et al.  Test-suite reduction and prioritization for modified condition/decision coverage , 2001, Proceedings IEEE International Conference on Software Maintenance. ICSM 2001.

[6]  Antonia Bertolino,et al.  Software Testing Research: Achievements, Challenges, Dreams , 2007, Future of Software Engineering (FOSE '07).

[7]  Lori A. Clarke,et al.  A System to Generate Test Data and Symbolically Execute Programs , 1976, IEEE Transactions on Software Engineering.

[8]  Annie S. Wu,et al.  A Genetic Algorithm Approach to Focused Software Usage Testing , 2003 .

[9]  Ron Patton,et al.  Software Testing , 2000 .

[10]  Donald J. Berndt,et al.  Investigating the performance of genetic algorithm-based software test case generation , 2004, Eighth IEEE International Symposium on High Assurance Systems Engineering, 2004. Proceedings..

[11]  Cem Kaner,et al.  Testing Computer Software , 1988 .

[12]  Jon Edvardsson,et al.  A Survey on Automatic Test Data Generation , 2002 .

[13]  Mark Harman,et al.  Evolutionary testing in the presence of loop-assigned flags: a testability transformation approach , 2004, ISSTA '04.

[14]  Clémentine Nebut,et al.  Automatic test generation: a use case driven approach , 2006, IEEE Transactions on Software Engineering.

[15]  Gregg Rothermel,et al.  Leveraging user-session data to support Web application testing , 2005, IEEE Transactions on Software Engineering.

[16]  Mark Harman,et al.  The Current State and Future of Search Based Software Engineering , 2007, Future of Software Engineering (FOSE '07).

[17]  Mark Harman,et al.  The species per path approach to SearchBased test data generation , 2006, ISSTA '06.

[18]  Hong Zhu,et al.  Software unit test coverage and adequacy , 1997, ACM Comput. Surv..

[19]  Sarfraz Khurshid,et al.  Exploring very large state spaces using genetic algorithms , 2004, International Journal on Software Tools for Technology Transfer.

[20]  Michael R. Lyu,et al.  Achieving software quality with testing coverage measures , 1994, Computer.

[21]  Fred W. Glover,et al.  Tabu Search - Part I , 1989, INFORMS J. Comput..

[22]  Corina S. Pasareanu,et al.  Test input generation for java containers using state matching , 2006, ISSTA '06.

[23]  Neelam Gupta,et al.  Generating test data for functions with pointer inputs , 2002, Proceedings 17th IEEE International Conference on Automated Software Engineering,.

[24]  André Baresel,et al.  Evolutionary Testing of Flag Conditions , 2003, GECCO.

[25]  Gregg Rothermel,et al.  Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis , 2004 .

[26]  Stephen R. Schach,et al.  Object-oriented and classical software engineering , 1995 .

[27]  Gary McGraw,et al.  Generating Software Test Data by Evolution , 2001, IEEE Trans. Software Eng..

[28]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[29]  Bogdan Korel,et al.  Automated test data generation for programs with procedures , 1996, ISSTA '96.

[30]  Yannis Smaragdakis,et al.  JCrasher: an automatic robustness tester for Java , 2004, Softw. Pract. Exp..

[31]  Sarfraz Khurshid,et al.  Generalized Symbolic Execution for Model Checking and Testing , 2003, TACAS.

[32]  Sarfraz Khurshid,et al.  Test input generation with java PathFinder , 2004, ISSTA '04.

[33]  David Notkin,et al.  Symstra: A Framework for Generating Object-Oriented Unit Tests Using Symbolic Execution , 2005, TACAS.

[34]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[35]  Pankaj Mudholkar,et al.  Software Testing , 2002, Computer.

[36]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[37]  Dawson R. Engler,et al.  EXE: automatically generating inputs of death , 2006, CCS '06.

[38]  Nikolai Tillmann,et al.  Parameterized unit tests , 2005, ESEC/FSE-13.

[39]  Gregg Rothermel.  ISSTA 2004 : proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis, Boston, Massachusetts, USA, July 11-14, 2004 , 2004 .

[40]  Mark Harman,et al.  Generating feasible input sequences for extended finite state machines (EFSMs) using genetic algorithms , 2005, GECCO '05.

[41]  Andreas S. Andreou,et al.  EXTENDING AND ENHANCING A BASIC PROGRAM ANALYSIS SYSTEM , 2006 .

[42]  Acm Sigsoft,et al.  ESEC/FSE'05 : proceedings of the joint 10th European Software Engineering Conference (ESEC) and the 13th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE-13), September 5-9, 2005, Lisbon, Portugal , 2005 .

[43]  Thomas Barr,et al.  Architectural overview of the computational intelligence testing tool , 2004, Eighth IEEE International Symposium on High Assurance Systems Engineering, 2004. Proceedings..

[44]  Leonardo Bottaci.  Instrumenting Programs With Flag Variables For Test Data Search By Genetic Algorithms , 2002, GECCO.

[45]  Gregg Rothermel,et al.  Integrating automated test generation into the WYSIWYT spreadsheet testing methodology , 2006, TSEM.

[46]  Koushik Sen.  DART: Directed Automated Random Testing , 2009, Haifa Verification Conference.

[47]  Roy P. Pargas,et al.  Test‐data generation using genetic algorithms , 1999 .

[48]  Paolo Tonella,et al.  Evolutionary testing of classes , 2004, ISSTA '04.

[49]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.

[50]  Michael D. Ernst,et al.  Eclat: Automatic Generation and Classification of Test Inputs , 2005, ECOOP.

[51]  Mark Harman,et al.  The Species per Path Approach to Search-Based Software Test Data Generation , 2006 .