JST: An automatic test generation tool for industrial Java applications with strings

In this paper we present JST, a tool that automatically generates a high coverage test suite for industrial strength Java applications. This tool uses a numeric-string hybrid symbolic execution engine at its core which is based on the Symbolic Java PathFinder platform. However, in order to make the tool applicable to industrial applications the existing generic platform had to be enhanced in numerous ways that we describe in this paper. The JST tool consists of newly supported essential Java library components and widely used data structures; novel solving techniques for string constraints, regular expressions, and their interactions with integer and floating point numbers; and key optimizations that make the tool more efficient. We present a methodology to seamlessly integrate the features mentioned above to make the tool scalable to industrial applications that are beyond the reach of the original platform in terms of both applicability and performance. We also present extensive experimental data to illustrate the effectiveness of our tool.

[1]  Nikolai Tillmann,et al.  Pex-White Box Test Generation for .NET , 2008, TAP.

[2]  Oscar H. Ibarra,et al.  Symbolic String Verification: Combining String Analysis and Size Analysis , 2009, TACAS.

[3]  D. Shannon,et al.  Efficient symbolic execution of strings for validating web applications , 2009, DEFECTS '09.

[4]  Gideon Redelinghuys,et al.  Symbolic string execution , 2012 .

[5]  Aske Simon Christensen,et al.  Precise Analysis of String Expressions , 2003, SAS.

[6]  Patrick Cousot,et al.  Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints , 1977, POPL.

[7]  Margus Veanes,et al.  An Evaluation of Automata Algorithms for String Analysis , 2011, VMCAI.

[8]  Michael R. Lowry,et al.  Combining unit-level symbolic execution and system-level concrete execution for testing nasa software , 2008, ISSTA '08.

[9]  Westley Weimer,et al.  Solving string constraints lazily , 2010, ASE.

[10]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[11]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[12]  Steve Hanna,et al.  A Symbolic Execution Framework for JavaScript , 2010, 2010 IEEE Symposium on Security and Privacy.

[13]  Guodong Li,et al.  KLOVER: A Symbolic Execution and Automatic Test Generation Tool for C++ Programs , 2011, CAV.

[14]  Marcelo d'Amorim,et al.  CORAL: Solving Complex Constraints for Symbolic PathFinder , 2011, NASA Formal Methods.

[15]  Willem Visser,et al.  Symbolic execution of programs with strings , 2012, SAICSIT '12.

[16]  Corina S. Pasareanu,et al.  Symbolic PathFinder: symbolic execution of Java bytecode , 2010, ASE.

[17]  Michael D. Ernst,et al.  HAMPI: A String Solver for Testing, Analysis and Vulnerability Detection , 2011, CAV.

[18]  Peng Li,et al.  GKLEE: concolic verification and test generation for GPUs , 2012, PPoPP '12.

[19]  Fang Yu,et al.  Stranger: An Automata-Based String Analysis Tool for PHP , 2010, TACAS.

[20]  Nikolaj Bjørner,et al.  Path Feasibility Analysis for String-Manipulating Programs , 2009, TACAS.

[21]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[22]  L. D. Moura,et al.  The YICES SMT Solver , 2006 .

[23]  Daniel Kroening,et al.  Decision Procedures - An Algorithmic Point of View , 2008, Texts in Theoretical Computer Science. An EATCS Series.

[24]  Margus Veanes,et al.  Rex: Symbolic Regular Expression Explorer , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.