Improving Constraint-Based Testing with Dynamic Linear Relaxations

Constraint-Based Testing (CBT) is the process of generating test cases against a testing objective by using constraint solving techniques. In CBT, testing objectives are given under the form of properties to be satisfied by program's input/output. Whenever the program or the properties contain disjunctions or multiplications between variables, CBT faces the problem of solving non-linear constraint systems. Currently, existing CBT tools tackle this problem by exploiting a finite-domains constraint solver. But, solving a non-linear constraint system over finite domains is NP hard and CBT tools fail to handle properly most properties to be tested. In this paper, we present a CBT approach where a finite domain constraint solver is enhanced by Dynamic Linear Relaxations (DLRs). DLRs are based on linear abstractions derived during the constraint solving process. They dramatically increase the solving capabilities of the solver in the presence of non-linear constraints without compromising the completeness or soundness of the overall CBT process. We implemented DLRs within the CBT tool TAUPO that generates test data for programs written in C. The approach has been validated on difficult non-linear properties over a few (academic) C programs.

[1]  Arnaud Gotlieb,et al.  Constraint-based test data generation in the presence of stack-directed pointers , 2005, ASE '05.

[2]  Matthew B. Dwyer,et al.  Finding feasible abstract counter-examples , 2003, International Journal on Software Tools for Technology Transfer.

[3]  Christian Holzbaur.  OFAI clp(Q,R) Manual , 1995 .

[4]  Mats Carlsson,et al.  An Open-Ended Finite Domain Constraint Solver , 1997, PLILP.

[5]  Mark N. Wegman,et al.  Efficiently computing static single assignment form and the control dependence graph , 1991, TOPL.

[6]  Bruno Marre,et al.  PathCrawler: Automatic Generation of Path Tests by Combining Static and Dynamic Analysis , 2005, EDCC.

[7]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[8]  Arnaud Gotlieb,et al.  Symbolic execution of floating‐point computations , 2006, Softw. Test. Verification Reliab..

[9]  Mary Lou Soffa,et al.  Automated test data generation using an iterative relaxation method , 1998, SIGSOFT '98/FSE-6.

[10]  Arnaud Gotlieb,et al.  Modelling Dynamic Memory Management in Constraint-Based Testing , 2007, Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007).

[11]  L. Fournier,et al.  Constraint satisfaction for test program generation , 1995, Proceedings International Phoenix Conference on Computers and Communications.

[12]  A. Jefferson Offutt,et al.  The Dynamic Domain Reduction Procedure for Test Data Generation: Design and Algorithms , 1994 .

[13]  A. Jefferson Offutt,et al.  The dynamic domain reduction procedure for test data generation , 1999, Softw. Pract. Exp..

[14]  A. Jefferson Offutt,et al.  Constraint-Based Automatic Test Data Generation , 1991, IEEE Trans. Software Eng..

[15]  Ioannis Parissis,et al.  Using CLP to Automatically Generate Test Sequences for Synchronous Programs with Numeric Inputs and Outputs , 2006, 2006 17th International Symposium on Software Reliability Engineering.

[16]  Arnaud Gotlieb,et al.  Goal-oriented test data generation for pointer programs , 2007, Inf. Softw. Technol..

[17]  Henny B. Sipma,et al.  Efficient Strongly Relational Polyhedral Analysis , 2006, VMCAI.

[18]  Nicolas Halbwachs,et al.  Automatic discovery of linear restraints among variables of a program , 1978, POPL.

[19]  Christophe Meudec.  ATGen: automatic test data generation using constraint logic programming and symbolic execution † , 2001, Softw. Test. Verification Reliab..

[20]  Daniel Jackson,et al.  Finding bugs with a constraint solver , 2000, ISSTA '00.

[21]  Neelam Gupta,et al.  Generating test data for functions with pointer inputs , 2002, Proceedings 17th IEEE International Conference on Automated Software Engineering,.

[22]  Garth P. McCormick,et al.  Computability of global solutions to factorable nonconvex programs: Part I — Convex underestimating problems , 1976, Math. Program..

[23]  James E. Falk,et al.  Jointly Constrained Biconvex Programming , 1983, Math. Oper. Res..

[24]  Arnaud Gotlieb,et al.  Automatic test data generation using constraint solving techniques , 1998, ISSTA '98.

[25]  Hélène Collavizza,et al.  Exploration of the Capabilities of Constraint Programming for Software Verification , 2006, TACAS.