Regression Test Suite Prioritization using Genetic Algorithms

Regression testing is an expensive, but important process in software testing. Unfortunately, there may be insufficient resources to allow for the re-execution of all test cases during regression testing. In this situation, test case prioritization techniques aim to improve the effectiveness of regression testing by ordering the test cases so that the most beneficial are executed first. In this paper we propose a new test case prioritization technique using Genetic Algorithm (GA). The proposed technique prioritizes subsequences of the original test suite so that the new suite, which is run within a time-constrained execution environment, will have a superior rate of fault detection when compared to rates of randomly prioritized test suites. This experiment analyzes the genetic algorithm with regard to effectiveness and time overhead by utilizing structurally-based criterion to prioritize test cases. An Average Percentage of Faults Detected (APFD) metric is used to determine the effectiveness of the new test case orderings.

[1]  Gregg Rothermel,et al.  Prioritizing test cases for regression testing , 2000, ISSTA '00.

[2]  Ivan Moore.  Jester - a JUnit test tester , 2001 .

[3]  Joseph Robert Horgan,et al.  A study of effective regression testing in practice , 1997, Proceedings The Eighth International Symposium on Software Reliability Engineering.

[4]  Adam A. Porter,et al.  A history-based test prioritization technique for regression testing in resource constrained environments , 2002, ICSE '02.

[5]  Ron Patton,et al.  Software Testing , 2000 .

[6]  Gregg Rothermel,et al.  Test Case Prioritization: A Family of Empirical Studies , 2002, IEEE Trans. Software Eng..

[7]  John E. Beasley,et al.  A Genetic Algorithm for the Multidimensional Knapsack Problem , 1998, J. Heuristics.

[8]  Gregg Rothermel,et al.  Incorporating varying test costs and fault severities into test case prioritization , 2001, Proceedings of the 23rd International Conference on Software Engineering. ICSE 2001.

[9]  Michael D. Ernst,et al.  An experimental evaluation of continuous testing during development , 2004, ISSTA '04.

[10]  Mark Harman,et al.  Evolving transformation sequences using genetic algorithms , 2004, Source Code Analysis and Manipulation, Fourth IEEE International Workshop on.

[11]  Gregg Rothermel,et al.  Empirical studies of test case prioritization in a JUnit testing environment , 2004, 15th International Symposium on Software Reliability Engineering.

[12]  Yves Ledru,et al.  Experiences in coverage testing of a Java middleware , 2005, SEM '05.

[13]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[14]  Charles J. Poole,et al.  Using Extreme Programming in a Maintenance Environment , 2001, IEEE Softw..

[15]  Mark Harman,et al.  Search Algorithms for Regression Test Case Prioritization , 2007, IEEE Transactions on Software Engineering.

[16]  Michael D. Ernst,et al.  Continuous testing in Eclipse , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[17]  Neelam Gupta,et al.  Test Case Prioritization Using Relevant Slices , 2006, 30th Annual International Computer Software and Applications Conference (COMPSAC'06).

[18]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[19]  Mary Lou Soffa,et al.  Jazz: A Tool for Demand-Driven Structural Testing , 2005, CC.

[20]  Amitabh Srivastava,et al.  Effectively prioritizing tests in development environment , 2002, ISSTA '02.

[21]  Gregg Rothermel,et al.  Test case prioritization: an empirical study , 1999, Proceedings IEEE International Conference on Software Maintenance - 1999 (ICSM'99). 'Software Maintenance for Business Change' (Cat. No.99CB36360).

[22]  Michael D. Ernst,et al.  Reducing wasted development time via continuous testing , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[23]  Giuliano Antoniol,et al.  Search-based techniques applied to optimization of project planning for a massive maintenance project , 2005, 21st IEEE International Conference on Software Maintenance (ICSM'05).

[24]  Gregg Rothermel,et al.  Prioritizing JUnit Test Cases: An Empirical Assessment and Cost-Benefits Analysis , 2006, Empirical Software Engineering.

[25]  Roy P. Pargas,et al.  Test-Data Generation Using Genetic Algorithms , 1999, Softw. Test. Verification Reliab..