Lightweight Data-Flow Analysis for Execution-Driven Constraint Solving

Constraint-based testing is a methodology for finding bugs in code, which has been successfully used for testing real systems. A key element of the methodology is generation of test inputs from input constraints, i.e., properties of desired inputs, which is performed by solving the constraints. We present a novel approach to optimize input generation from imperative constraints, i.e., constraints written as predicates in an imperative language. A well known technique for solving such constraints is execution-driven monitoring, where the given predicate is executed on candidate inputs to filter and prune invalid inputs, and generate valid ones. Our insight is that a lightweight static data-flow analysis of the given imperative constraint can enable more efficient solving. This paper describes an approach that embodies our insight and evaluates it using a suite of well-studied subject constraints. The experimental results show our approach provides substantial speedup over previous work.

[1]  Sarfraz Khurshid,et al.  Optimizing Generation of Object Graphs in Java PathFinder , 2009, 2009 International Conference on Software Testing Verification and Validation.

[2]  Sarfraz Khurshid,et al.  Test generation through programming in UDITA , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[3]  Klaus Havelund,et al.  Model Checking Programs , 2004, Automated Software Engineering.

[4]  Daniel Jackson,et al.  Alloy: a lightweight object modelling notation , 2002, TSEM.

[5]  Daniel Kroening,et al.  A Tool for Checking ANSI-C Programs , 2004, TACAS.

[6]  Lori A. Clarke,et al.  Test data generation and symbolic execution of programs as an aid to program validation. , 1976 .

[7]  Michael L. Fredman,et al.  The AETG System: An Approach to Testing Based on Combinatiorial Design , 1997, IEEE Trans. Software Eng..

[8]  Keith Stobie Model Based Testing in Practice at Microsoft , 2005, Electron. Notes Theor. Comput. Sci..

[9]  Sarfraz Khurshid,et al.  Efficient solving of structural constraints , 2008, ISSTA '08.

[10]  Sarfraz Khurshid,et al.  Generalized Symbolic Execution for Model Checking and Testing , 2003, TACAS.

[11]  Sarfraz Khurshid,et al.  Software assurance by bounded exhaustive testing , 2004, IEEE Transactions on Software Engineering.

[12]  Sarfraz Khurshid,et al.  TestEra: a novel framework for automated testing of Java programs , 2001, Proceedings 16th Annual International Conference on Automated Software Engineering (ASE 2001).

[13]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[14]  Dawson R. Engler,et al.  EXE: automatically generating inputs of death , 2006, CCS '06.

[15]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[16]  Vikram S. Adve,et al.  LLVA: A Low-level Virtual Instruction Set Architecture , 2003, MICRO.

[17]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.

[18]  Sarfraz Khurshid,et al.  Starc: static analysis for efficient repair of complex data , 2007, OOPSLA.

[19]  C. V. Ramamoorthy,et al.  On the Automated Generation of Program Test Data , 1976, IEEE Transactions on Software Engineering.

[20]  Koushik Sen,et al.  DART: directed automated random testing , 2005, PLDI '05.

[21]  Chandrasekhar Boyapati,et al.  Efficient software model checking of data structure properties , 2006, OOPSLA '06.

[22]  Sarfraz Khurshid,et al.  Parallel test generation and execution with Korat , 2007, ESEC-FSE '07.

[23]  Sarfraz Khurshid,et al.  PKorat: Parallel Generation of Structurally Complex Test Inputs , 2009, 2009 International Conference on Software Testing Verification and Validation.

[24]  Dawson R. Engler,et al.  Practical, Low-Effort Equivalence Verification of Real Code , 2011, CAV.

[25]  Sarfraz Khurshid,et al.  Optimizing a Structural Constraint Solver for Efficient Software Checking , 2009, 2009 IEEE/ACM International Conference on Automated Software Engineering.

[26]  Daniel Jackson,et al.  Software Abstractions - Logic, Language, and Analysis , 2006 .

[27]  Darko Marinov,et al.  Automatic testing of software with structurally complex inputs , 2005 .

[28]  Sarfraz Khurshid,et al.  An Empirical Study of Structural Constraint Solving Techniques , 2009, ICFEM.

[29]  J. C. Huang,et al.  An Approach to Program Testing , 1975, CSUR.