An Improved Memetic Algorithm with Method Dependence Relations (MAMDR)

Search-based approaches are successfully used for generating unit tests for object-oriented programs in Java. However, these approaches may struggle to generate sequence method calls with specific values to achieve high coverage due to the large size of the search space. This paper proposes a memetic algorithm (MA) approach in which static analysis is used to identify method dependence relations (MDR) based on the field access. This method dependence information is employed for reducing the search space and used to guide the search towards regions that lead to full (or at least high) structural coverage. Our approach, MAMDR, combines both a genetic algorithm (GA) and Hill Climbing (HC) to generate test data for Java programs. The former is used to produce test cases that maximize the branch coverage of the CUT, while minimizing the length of each test case. The latter is used to target uncovered branches in the preceding search phase using static information that guides the search to generate sequences of method calls and values that could cover target branches. We compare MAMDR with pure random testing, a well-known search based approach (EvoSuite), and a simple MA on several open source projects and classes, and show that the combination of MA and MDR is effective.

[1]  Zhendong Su,et al.  Synthesizing method sequences for high-coverage testing , 2011, OOPSLA '11.

[2]  Bertrand Meyer,et al.  Experimental assessment of random testing for object-oriented software , 2007, ISSTA '07.

[3]  Francisco Fernández de Vega,et al.  Test Case Evaluation and Input Domain Reduction strategies for the Evolutionary Testing of Object-Oriented software , 2009, Inf. Softw. Technol..

[4]  Gordon Fraser,et al.  Improving search-based test suite generation with dynamic symbolic execution , 2013, 2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE).

[5]  Phil McMinn,et al.  Search-Based Test Input Generation for String Data Types Using the Results of Web Queries , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[6]  Xin Yao,et al.  A Memetic Algorithm for test data generation of Object-Oriented software , 2007, 2007 IEEE Congress on Evolutionary Computation.

[7]  Michael D. Ernst,et al.  Dependence-Guided Random Test Generation , 2010 .

[8]  Gordon Fraser,et al.  Testing Container Classes: Random or Systematic? , 2011, FASE.

[9]  André Baresel,et al.  Fitness Function Design To Improve Evolutionary Structural Testing , 2002, GECCO.

[10]  Gordon Fraser,et al.  Semi-automatic Search-Based Test Generation , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[11]  Bogdan Korel,et al.  Automated Software Test Data Generation , 1990, IEEE Trans. Software Eng..

[12]  Gary McGraw,et al.  Generating Software Test Data by Evolution , 2001, IEEE Trans. Software Eng..

[13]  Luciano Baresi,et al.  TestFul: An Evolutionary Test Approach for Java , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[14]  Paolo Tonella,et al.  Evolutionary testing of classes , 2004, ISSTA '04.

[15]  Gordon Fraser,et al.  Test suite generation with memetic algorithms , 2013, GECCO '13.

[16]  Mark Harman,et al.  Testability transformation , 2004, IEEE Transactions on Software Engineering.

[17]  Gordon Fraser,et al.  The Seed is Strong: Seeding Strategies in Search-Based Software Testing , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[18]  Hans-Gerhard Groß,et al.  A Genetic Programming Approach to Automated Test Generation for Object-Oriented Software , 2006, Int. Trans. Syst. Sci. Appl..

[19]  Tao Xie,et al.  Improving Structural Testing of Object-Oriented Programs via Integrating Evolutionary Testing and Symbolic Execution , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[20]  Gordon Fraser,et al.  Whole Test Suite Generation , 2013, IEEE Transactions on Software Engineering.

[21]  Alex Groce.  Coverage rewarded: Test input generation via adaptation-based programming , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[22]  Andrea Arcuri,et al.  It really does matter how you normalize the branch distance in search‐based software testing , 2013, Softw. Test. Verification Reliab..

[23]  Stefan Wappler,et al.  Using evolutionary algorithms for the unit testing of object-oriented software , 2005, GECCO '05.

[24]  Phil McMinn,et al.  Search‐based software test data generation: a survey , 2004, Softw. Test. Verification Reliab..

[25]  Michael D. Ernst,et al.  Feedback-Directed Random Test Generation , 2007, 29th International Conference on Software Engineering (ICSE'07).

[26]  Mark Harman,et al.  Automated test data generation for aspect-oriented programs , 2009, AOSD '09.

[27]  Joachim Wegener,et al.  Evolutionary test environment for automatic structural testing , 2001, Inf. Softw. Technol..

[28]  Martin Rinard,et al.  A Combined Pointer and Purity Analysis for Java Programs , 2004 .

[29]  Mark Harman,et al.  The impact of input domain reduction on search-based test data generation , 2007, ESEC-FSE '07.

[30]  Mark Harman,et al.  Analysis and visualization of predicate dependence on formal parameters and global variables , 2004, IEEE Transactions on Software Engineering.

[31]  Bogdan Korel,et al.  The chaining approach for software test data generation , 1996, TSEM.

[32]  Gordon Fraser,et al.  Handling test length bloat , 2013, Softw. Test. Verification Reliab..

[33]  Phil McMinn,et al.  Hybridizing Evolutionary Testing with the Chaining Approach , 2004, GECCO.

[34]  Lionel C. Briand,et al.  A practical guide for using statistical tests to assess randomized algorithms in software engineering , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[35]  Alex Groce,et al.  Help, help, i'm being suppressed! The significance of suppressors in software testing , 2013, 2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE).

[36]  Laurie Hendren,et al.  Soot: a Java bytecode optimization framework , 2010, CASCON.

[37]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[38]  Mohammad Alshraideh,et al.  Search‐based software test data generation for string data using program‐specific search operators , 2006, Softw. Test. Verification Reliab..

[39]  Gordon Fraser,et al.  Parameter tuning or default values? An empirical investigation in search-based software engineering , 2013, Empirical Software Engineering.

[40]  David W. Binkley,et al.  Program slicing , 2008, 2008 Frontiers of Software Maintenance.

[41]  Mark Harman,et al.  Automated web application testing using search based software engineering , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[42]  Michael D. Ernst,et al.  Combined static and dynamic automated test generation , 2011, ISSTA '11.