Directed Symbolic Execution

In this paper, we study the problem of automatically finding program executions that reach a particular target line. This problem arises in many debugging scenarios; for example, a developer may want to confirm that a bug reported by a static analysis tool on a particular line is a true positive. We propose two new directed symbolic execution strategies that aim to solve this problem: shortest-distance symbolic execution (SDSE) uses a distance metric in an interprocedural control flow graph to guide symbolic execution toward a particular target; and call-chain-backward symbolic execution (CCBSE) iteratively runs forward symbolic execution, starting in the function containing the target line, and then jumping backward up the call chain until it finds a feasible path from the start of the program. We also propose a hybrid strategy, Mix-CCBSE, which alternates CCBSE with another (forward) search strategy. We compare these three with several existing strategies from the literature on a suite of six GNU Coreutils programs. We find that SDSE performs extremely well in many cases but may fail badly. CCBSE also performs quite well, but imposes additional overhead that sometimes makes it slower than SDSE. Considering all our benchmarks together, Mix-CCBSE performed best on average, combining to good effect the features of its constituent components.

[1]  Karl N. Levitt,et al.  SELECT—a formal system for testing and debugging programs by symbolic execution , 1975 .

[2]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[3]  Leon J. Osterweil,et al.  Program testing techniques using simulated execution , 1976, ANSS '76.

[4]  William E. Howden,et al.  Symbolic Testing and the DISSECT Symbolic Evaluation System , 1977, IEEE Transactions on Software Engineering.

[5]  Joseph M. Morris.  A General Axiom of Assignment , 1982 .

[6]  Joseph M. Morris.  A Proof of the Schorr-Waite Algorithm , 1982 .

[7]  Barbara G. Ryder,et al.  Pointer-induced aliasing: a problem classification , 1991, POPL '91.

[8]  Thomas W. Reps,et al.  Program analysis via graph reachability , 1997, Inf. Softw. Technol..

[9]  Jakob Rehof,et al.  Scalable context-sensitive flow analysis using instantiation constraints , 2000, PLDI '00.

[10]  Richard Bornat,et al.  Proving Pointer Programs in Hoare Logic , 2000, MPC.

[11]  Anna Philippou,et al.  Tools and Algorithms for the Construction and Analysis of Systems , 2018, Lecture Notes in Computer Science.

[12]  Jakob Rehof,et al.  Type-base flow analysis: from polymorphic subtyping to CFL-reachability , 2001, POPL '01.

[13]  Stefan Leue,et al.  Trail-directed model checking , 2001, Workshop on Software Model Checking @ CAV.

[14]  George C. Necula,et al.  CIL: Intermediate Language and Tools for Analysis and Transformation of C Programs , 2002, CC.

[15]  Stefan Edelkamp,et al.  Directed explicit-state model checking in the validation of communication protocols , 2004, International Journal on Software Tools for Technology Transfer.

[16]  Alex Groce,et al.  Model checking Java programs using structural heuristics , 2002, ISSTA '02.

[17]  Sarfraz Khurshid,et al.  Generalized Symbolic Execution for Model Checking and Testing , 2003, TACAS.

[18]  Alexander Aiken,et al.  The set constraint/CFL reachability connection in practice , 2004, PLDI '04.

[19]  Vikram S. Adve,et al.  LLVM: a compilation framework for lifelong program analysis & transformation , 2004, International Symposium on Code Generation and Optimization, 2004. CGO 2004..

[20]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[21]  S. Kupferschmid,et al.  Adapting an AI Planning Heuristic for Directed Model Checking , 2006, SPIN.

[22]  Bruno Dutertre,et al.  A Fast Linear-Arithmetic Solver for DPLL(T) , 2006, CAV.

[23]  Patrice Godefroid,et al.  Compositional dynamic test generation , 2007, POPL '07.

[24]  Rupak Majumdar,et al.  Directed test generation using symbolic grammars , 2007, ESEC-FSE companion '07.

[25]  David L. Dill,et al.  A Decision Procedure for Bit-Vectors and Arrays , 2007, CAV.

[26]  Rupak Majumdar,et al.  Hybrid Concolic Testing , 2007, 29th International Conference on Software Engineering (ICSE'07).

[27]  Andreas Podelski,et al.  ACSAR: Software Model Checking with Transfinite Refinement , 2007, SPIN.

[28]  Dawson R. Engler,et al.  EXE: Automatically Generating Inputs of Death , 2008, TSEC.

[29]  Patrice Godefroid,et al.  Automated Whitebox Fuzz Testing , 2008, NDSS.

[30]  Patrice Godefroid,et al.  Active property checking , 2008, EMSOFT '08.

[31]  Koushik Sen,et al.  Heuristics for Scalable Dynamic Test Generation , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[32]  Adam Kiezun,et al.  Grammar-based whitebox fuzzing , 2008, PLDI '08.

[33]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[34]  Dawson R. Engler,et al.  RWset: Attacking Path Explosion in Constraint-Based Test Generation , 2008, TACAS.

[35]  Stephen McCamant,et al.  Loop-extended symbolic execution on binary programs , 2009, ISSTA.

[36]  Nikolai Tillmann,et al.  Fitness-guided path exploration in dynamic symbolic execution , 2009, 2009 IEEE/IFIP International Conference on Dependable Systems & Networks.

[37]  Patrice Godefroid,et al.  Precise pointer reasoning for dynamic test generation , 2009, ISSTA.

[38]  Frank Piessens,et al.  Test Input Generation for Programs with Pointers , 2009, TACAS.

[39]  Koushik Sen.  DART: Directed Automated Random Testing , 2009, Haifa Verification Conference.

[40]  George Candea,et al.  Execution synthesis: a technique for automated software debugging , 2010, EuroSys '10.

[41]  Bor-Yuh Evan Chang,et al.  Mixing type checking and symbolic execution , 2010, PLDI '10.

[42]  Adam A. Porter,et al.  Using symbolic evaluation to understand behavior in configurable software systems , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.