LLSPLAT: Improving Concolic Testing by Bounded Model Checking

For software testing, concolic testing reasons about data symbolically but enumerates program paths. The existing concolic technique enumerates paths sequentially, leading to poor branch coverage in limited time. In this paper, we improve concolic testing by bounded model checking (BMC). During concolic testing, we identify program regions that can be encoded by BMC on the fly so that program paths within these regions are checked simultaneously. We have implemented the new algorithm on top of KLEE and called the new tool LLSPLAT. We have compared LLSPLAT with KLEE using 10 programs from the Windows NT Drivers Simplified and 88 programs from the GNU Coreutils benchmark sets. With 3600 second testing time for each program, LLSPLAT provides on average 13% relative branch coverage improvement on all 10 programs in the Windows drivers set, and on average 16% relative branch coverage improvement on 80 out of 88 programs in the GNU Coreutils set.

[1]  Daniel Kroening,et al.  Loop summarization using state and transition invariants , 2013, Formal Methods Syst. Des..

[2]  Cormac Flanagan,et al.  Avoiding exponential explosion: generating compact verification conditions , 2001, POPL '01.

[3]  Nikolai Tillmann,et al.  Fitness-guided path exploration in dynamic symbolic execution , 2009, 2009 IEEE/IFIP International Conference on Dependable Systems & Networks.

[4]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[5]  Koushik Sen DART: Directed Automated Random Testing , 2009, Haifa Verification Conference.

[6]  Dirk Beyer,et al.  Software model checking via large-block encoding , 2009, 2009 Formal Methods in Computer-Aided Design.

[7]  Dawson R. Engler,et al.  RWset: Attacking Path Explosion in Constraint-Based Test Generation , 2008, TACAS.

[8]  Thomas A. Henzinger,et al.  Abstraction-driven Concolic Testing , 2015, VMCAI.

[9]  Manu Sridharan,et al.  Snugglebug: a powerful approach to weakest preconditions , 2009, PLDI '09.

[10]  William R. Harris,et al.  Program analysis via satisfiability modulo path programs , 2010, POPL '10.

[11]  Nikolai Tillmann,et al.  Demand-Driven Compositional Symbolic Execution , 2008, TACAS.

[12]  Daniel Kroening,et al.  Behavioral consistency of C and Verilog programs using bounded model checking , 2003, Proceedings 2003. Design Automation Conference (IEEE Cat. No.03CH37451).

[13]  George Candea,et al.  Efficient state merging in symbolic execution , 2012, Software Engineering.

[14]  Koushik Sen,et al.  Heuristics for Scalable Dynamic Test Generation , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[15]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[16]  Zhendong Su,et al.  Steering symbolic execution to less traveled paths , 2013, OOPSLA.

[17]  Ting Chen,et al.  State of the art: Dynamic symbolic execution for automated test generation , 2013, Future Gener. Comput. Syst..

[18]  Patrice Godefroid,et al.  Compositional dynamic test generation , 2007, POPL '07.

[19]  Carsten Sinz,et al.  LLBMC: Bounded Model Checking of C and C++ Programs Using a Compiler IR , 2012, VSTTE.

[20]  K. Rustan M. Leino,et al.  Weakest-precondition of unstructured programs , 2005, PASTE '05.

[21]  Jorge A. Navas,et al.  Boosting concolic testing via interpolation , 2013, ESEC/FSE 2013.

[22]  Dirk Beyer,et al.  Competition on Software Verification - (SV-COMP) , 2012, TACAS.

[23]  Koushik Sen,et al.  Symbolic execution for software testing: three decades later , 2013, CACM.

[24]  Rupak Majumdar,et al.  Hybrid Concolic Testing , 2007, 29th International Conference on Software Engineering (ICSE'07).

[25]  Lucas C. Cordeiro,et al.  Context-Bounded Model Checking with ESBMC 1.17 - (Competition Contribution) , 2012, TACAS.

[26]  Armin Biere,et al.  Bounded Model Checking Using Satisfiability Solving , 2001, Formal Methods Syst. Des..

[27]  Myra B. Cohen,et al.  An orchestrated survey of methodologies for automated software test case generation , 2013, J. Syst. Softw..

[28]  Shuvendu K. Lahiri,et al.  A Solver for Reachability Modulo Theories , 2012, CAV.

[29]  David Brumley,et al.  Enhancing symbolic execution with veritesting , 2014, ICSE.

[30]  Pablo Sánchez Espeso,et al.  An Approach to Static-Dynamic Software Analysis , 2015, FTSCS.

[31]  Sunghun Kim,et al.  How we get there: a context-guided search strategy in concolic testing , 2014, SIGSOFT FSE.

[32]  Raúl A. Santelices,et al.  Exploiting program dependencies for scalable multiple-path symbolic execution , 2010, ISSTA '10.