Path-oriented test data generation using symbolic execution and constraint solving techniques

Automatic test data generation is a challenging task in software engineering research. This paper studies a path-oriented approach to the problem, which is based on the combination of symbolic execution and constraint solving. Methods for representing expressions and path conditions are discussed. An implemented toolkit is described with some examples. The toolkit transforms an input program (possibly embedded with assertions) to an extended finite state machine and then performs depth-first or breadth-first search on it. The goal is to find values for input variables such that a terminal state can be reached. If successful, input test data are found (which might reveal a bug in the program).

[1]  Jian Zhang,et al.  Symbolic execution of program paths involving pointer structure variables , 2004, Fourth International Conference onQuality Software, 2004. QSIC 2004. Proceedings..

[2]  Karl N. Levitt,et al.  SELECT—a formal system for testing and debugging programs by symbolic execution , 1975 .

[3]  William R. Bush,et al.  A static analyzer for finding dynamic programming errors , 2000, Softw. Pract. Exp..

[4]  Carlo Ghezzi,et al.  Using symbolic execution for verifying safety-critical systems , 2001, ESEC/FSE-9.

[5]  Bogdan Korel,et al.  Assertion-oriented automated test data generation , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[6]  Roy P. Pargas,et al.  Test‐data generation using genetic algorithms , 1999, Softw. Test. Verification Reliab..

[7]  Tad Hogg,et al.  Phase Transitions and the Search Problem , 1996, Artif. Intell..

[8]  P. D. Coward,et al.  Symbolic execution and testing , 1990 .

[9]  A. Jefferson Offutt,et al.  The dynamic domain reduction procedure for test data generation , 1999, Softw. Pract. Exp..

[10]  Antonia Bertolino,et al.  Automatic Generation of Path Covers Based on the Control Flow Analysis of Computer Programs , 1994, IEEE Trans. Software Eng..

[11]  A. Jefferson Offutt,et al.  Constraint-Based Automatic Test Data Generation , 1991, IEEE Trans. Software Eng..

[12]  Jian Zhang.  Symbolic execution of program paths involving pointer structure variables , 2004 .

[13]  David Notkin,et al.  Model checking large software specifications , 1996, SIGSOFT '96.

[14]  Richard A. Kemmerer,et al.  Unisex: A unix‐based symbolic executor for pascal , 1985, Softw. Pract. Exp..

[15]  R. Dechter.  to Constraint Satisfaction , 1991 .

[16]  Greg Nelson,et al.  Extended static checking for Java , 2002, PLDI '02.

[17]  Sorin Lerner,et al.  ESP: path-sensitive program verification in polynomial time , 2002, PLDI '02.

[18]  Arnaud Gotlieb,et al.  Automatic test data generation using constraint solving techniques , 1998, ISSTA '98.

[19]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[20]  Mary Lou Soffa,et al.  Automated test data generation using an iterative relaxation method , 1998, SIGSOFT '98/FSE-6.

[21]  Lori A. Clarke,et al.  A System to Generate Test Data and Symbolically Execute Programs , 1976, IEEE Transactions on Software Engineering.

[22]  Jian Zhang,et al.  A Constraint Solver and Its Application to Path Feasibility Analysis , 2001, Int. J. Softw. Eng. Knowl. Eng..

[23]  Martin R. Woodward,et al.  Experience with Path Analysis and Testing of Programs , 1980, IEEE Transactions on Software Engineering.

[24]  Laura K. Dillon,et al.  Using symbolic execution for verification of Ada tasking programs , 1990, TOPL.

[25]  Alessandro Orso,et al.  Automated Testing of Classes , 2000, ISSTA '00.

[26]  Gary McGraw,et al.  Generating Software Test Data by Evolution , 2001, IEEE Trans. Software Eng..

[27]  A. Jefferson Offutt,et al.  The dynamic domain reduction procedure for test data generation , 1999, Softw. Pract. Exp..

[28]  William R. Bush,et al.  A static analyzer for finding dynamic programming errors , 2000 .

[29]  Tsong Yueh Chen,et al.  Semi-proving: an integrated method based on global symbolic evaluation and metamorphic testing , 2002, ISSTA '02.

[30]  John A. Clark,et al.  Automated program flaw finding using simulated annealing , 1998, ISSTA '98.

[31]  Nicos Malevris,et al.  Reducing the Effects of Infeasible Paths in Branch Testing , 1989, Symposium on Testing, Analysis, and Verification.

[32]  Barton P. Miller,et al.  An empirical study of the reliability of UNIX utilities , 1990, Commun. ACM.