Directed test suite augmentation via exploiting program dependency

Regression testing is a practice to discover faults introduced in the program modification. However, the existing test suite is usually designed at an early stage of software development and is therefore insensitive to subsequent program changes. Although symbolic execution such as JPF-SE is able to produce test cases for the modified program by exhaustively exploring program paths, it is not tailored for testing program changes and is not scalable. In this paper, we propose an efficient approach called Directed Test Suite Augmentation (DTSA) to automatically generate test cases that can reach the changed statements, produce different program states after executing the changed statements, and propagate the different states to the output of the program. The key insight of DTSA is to reorder the generated test cases tailored for testing program changes compared with JPF-SE. We implemented a prototype of our approach and the experiment results show that DTSA requires about 60.1% and 45.6% fewer Dynamic Symbolic Execution (DSE) runs to generate the desired test case than JPF-SE and another test suite augmentation tool eXpress, respectively.

[1]  Matthew B. Dwyer,et al.  A new foundation for control dependence and slicing for modern program structures , 2005, TOPL.

[2]  Jeffrey M. Voas,et al.  PIE: A Dynamic Failure-Based Technique , 1992, IEEE Trans. Software Eng..

[3]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[4]  Thomas W. Reps,et al.  Precise interprocedural dataflow analysis via graph reachability , 1995, POPL '95.

[5]  Alessandro Orso,et al.  Test-Suite Augmentation for Evolving Software , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[6]  Sarfraz Khurshid,et al.  Directed incremental symbolic execution , 2011, PLDI '11.

[7]  Koushik Sen,et al.  DART: directed automated random testing , 2005, PLDI '05.

[8]  Corina S. Pasareanu,et al.  Symbolic PathFinder: symbolic execution of Java bytecode , 2010, ASE.

[9]  Thomas J. Ostrand,et al.  Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria , 1994, Proceedings of 16th International Conference on Software Engineering.

[10]  Zhihong Xu,et al.  Directed test suite augmentation , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[11]  Derek Rayside,et al.  A generic worklist algorithm for graph reachability problems in program analysis , 2002, Proceedings of the Sixth European Conference on Software Maintenance and Reengineering.

[12]  Myra B. Cohen,et al.  Directed test suite augmentation: techniques and tradeoffs , 2010, FSE '10.

[13]  Zhenkai Liang,et al.  Test generation to expose changes in evolving programs , 2010, ASE '10.

[14]  Nikolai Tillmann,et al.  eXpress: guided path exploration for efficient regression test generation , 2011, ISSTA '11.