A Comparative Study of Incremental Constraint Solving Approaches in Symbolic Execution

Constraint solving is a major source of cost in Symbolic Execution (SE). This paper presents a study to assess the importance of some sensible options for solving constraints in SE. The main observation is that stack-based approaches to incremental solving is often much faster compared to cache-based approaches, which are more popular. Considering all 96 C programs from the KLEE benchmark that we analyzed, the median speedup obtained with a (non-optimized) stack-based approach was of 5x. Results suggest that tools should take advantage of incremental solving support from modern SMT solvers and researchers should look for ways to combine stack- and cache-based approaches to reduce execution cost even further. Instructions to reproduce results are available online: http://asa.iti.kit.edu/130_392.php

[1]  Milan Banković.  ArgoSMTExpression : an SMT-LIB 2 . 0 compliant expression library , 2012 .

[2]  Marsha Chechik,et al.  Symbolic optimization with SMT solvers , 2014, POPL.

[3]  Matthew B. Dwyer,et al.  Green: reducing, reusing and recycling constraints in program analysis , 2012, SIGSOFT FSE.

[4]  Matti Järvisalo,et al.  Theory and Applications of Satisfiability Testing – SAT 2013 , 2013, Lecture Notes in Computer Science.

[5]  Corina S. Pasareanu,et al.  A survey of new trends in symbolic execution for software testing and analysis , 2009, International Journal on Software Tools for Technology Transfer.

[6]  Mana Taghdiri,et al.  Bounded Program Verification Using an SMT Solver: A Case Study , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[7]  Wilfried Steiner,et al.  An Evaluation of SMT-Based Schedule Synthesis for Time-Triggered Multi-hop Networks , 2010, 2010 31st IEEE Real-Time Systems Symposium.

[8]  C. V. Ramamoorthy,et al.  On the Automated Generation of Program Test Data , 1976, IEEE Transactions on Software Engineering.

[9]  Gilles Audemard,et al.  Improving Glucose for Incremental SAT Solving with Assumptions: Application to MUS Extraction , 2013, SAT.

[10]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[11]  Siert Wieringa,et al.  Incremental Satisfiability Solving and its Applications , 2014 .

[12]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[13]  Lori A. Clarke,et al.  A program testing system , 1976, ACM '76.

[14]  Karl N. Levitt,et al.  SELECT—a formal system for testing and debugging programs by symbolic execution , 1975 .

[15]  Sarfraz Khurshid,et al.  Memoise: A tool for memoized symbolic execution , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[16]  David Brumley,et al.  All You Ever Wanted to Know about Dynamic Taint Analysis and Forward Symbolic Execution (but Might Have Been Afraid to Ask) , 2010, 2010 IEEE Symposium on Security and Privacy.

[17]  William E. Howden,et al.  Symbolic Testing and the DISSECT Symbolic Evaluation System , 1977, IEEE Transactions on Software Engineering.

[18]  Corina S. Pasareanu,et al.  Symbolic PathFinder: symbolic execution of Java bytecode , 2010, ASE.

[19]  Nikolai Tillmann,et al.  Pex-White Box Test Generation for .NET , 2008, TAP.

[20]  Sarfraz Khurshid,et al.  Symbolic execution for software testing in practice: preliminary assessment , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[21]  Marcelo d'Amorim,et al.  Compositional solution space quantification for probabilistic software analysis , 2014, PLDI.

[22]  Chen Fu,et al.  Evaluating program analysis and testing tools with the RUGRAT random benchmark application generator , 2012, WODA 2012.

[23]  Koushik Sen,et al.  Heuristics for Scalable Dynamic Test Generation , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.