Conc-iSE: Incremental symbolic execution of concurrent software

Software updates often introduce new bugs to existing code bases. Prior regression testing tools focus mainly on test case selection and prioritization whereas symbolic execution tools only handle code changes in sequential software. In this paper, we propose the first incremental symbolic execution method for concurrent software to generate new tests by exploring only the executions affected by code changes between two program versions. Specifically, we develop an inter-thread and inter-procedural change-impact analysis to check if a statement is affected by the changes and then leverage the information to choose executions that need to be re-explored. We also check if execution summaries computed in the previous program can be used to avoid redundant explorations in the new program. We have implemented our method in an incremental symbolic execution tool called Conc-iSE and evaluated it on a large set of multithreaded C programs. Our experiments show that the new method can significantly reduce the overall symbolic execution time when compared with state-of-the-art symbolic execution tools such as KLEE.

[1]  Alexander Aiken,et al.  Conditional must not aliasing for static race detection , 2007, POPL '07.

[2]  Chao Wang,et al.  Contessa: Concurrency Testing Augmented with Symbolic Analysis , 2010, CAV.

[3]  David W. Binkley,et al.  Program slicing , 2008, 2008 Frontiers of Software Maintenance.

[4]  Alexander Aiken,et al.  Effective static race detection for Java , 2006, PLDI '06.

[5]  Chao Wang,et al.  Assertion guided abstraction: a cooperative optimization for dynamic partial order reduction , 2014, ASE.

[6]  Chao Wang,et al.  Symbolic predictive analysis for concurrent programs , 2009, Formal Aspects of Computing.

[7]  George Candea,et al.  Cloud9: a software testing service , 2010, OPSR.

[8]  Chao Wang,et al.  On interference abstractions , 2011, POPL '11.

[9]  Jorge A. Navas,et al.  Boosting concolic testing via interpolation , 2013, ESEC/FSE 2013.

[10]  Chao Wang,et al.  Dynamic Analysis and Debugging of Binary Code for Security Applications , 2013, RV.

[11]  Monica S. Lam,et al.  Cloning-based context-sensitive pointer alias analysis using binary decision diagrams , 2004, PLDI '04.

[12]  Nikolaj Bjørner,et al.  μZ- An Efficient Engine for Fixed Points with Constraints , 2011, CAV.

[13]  Yannis Smaragdakis,et al.  Strictly declarative specification of sophisticated points-to analyses , 2009, OOPSLA '09.

[14]  Joe D. Warren,et al.  The program dependence graph and its use in optimization , 1987, TOPL.

[15]  Patrice Godefroid,et al.  Partial-Order Methods for the Verification of Concurrent Systems , 1996, Lecture Notes in Computer Science.

[16]  Joxan Jaffar,et al.  A Framework to Synergize Partial Order Reduction with State Interpolation , 2014, Haifa Verification Conference.

[17]  Steffen Lehnert,et al.  A taxonomy for software change impact analysis , 2011, IWPSE-EVOL '11.

[18]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[19]  Sarfraz Khurshid,et al.  Directed incremental symbolic execution , 2011, PLDI '11.

[20]  Edsger W. Dijkstra,et al.  A Discipline of Programming , 1976 .

[21]  Chao Wang,et al.  Assertion guided symbolic execution of multithreaded programs , 2015, ESEC/SIGSOFT FSE.

[22]  Wuu Yang,et al.  Identifying syntactic differences between two programs , 1991, Softw. Pract. Exp..

[23]  Helmut Veith,et al.  Con2colic testing , 2013, ESEC/FSE 2013.

[24]  Chao Wang,et al.  Staged concurrent program analysis , 2010, FSE '10.

[25]  Shing-Chi Cheung,et al.  RECONTEST: Effective Regression Testing of Concurrent Programs , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[26]  Brendan Murphy,et al.  The Art of Testing Less without Sacrificing Quality , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[27]  Suzette Person,et al.  A change impact analysis to characterize evolving program behaviors , 2012, 2012 28th IEEE International Conference on Software Maintenance (ICSM).

[28]  Kenneth L. McMillan.  Lazy Annotation for Program Testing and Verification , 2010, CAV.

[29]  Julian Dolby,et al.  Marathon: Detecting Atomic-Set Serializability Violations with Conflict Graphs , 2011, RV.

[30]  Sarfraz Khurshid,et al.  Property differencing for incremental checking , 2014, ICSE.

[31]  Darko Marinov,et al.  Change-aware preemption prioritization , 2011, ISSTA '11.

[32]  Jian Liu,et al.  Postconditioned Symbolic Execution , 2015, 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST).

[33]  Patrice Godefroid,et al.  Dynamic partial-order reduction for model checking software , 2005, POPL '05.

[34]  Chao Wang,et al.  Universal Causality Graphs: A Precise Happens-Before Model for Detecting Bugs in Concurrent Programs , 2010, CAV.

[35]  Shuvendu K. Lahiri,et al.  Differential assertion checking , 2013, ESEC/FSE 2013.

[36]  David W. Binkley,et al.  Interprocedural slicing using dependence graphs , 1990, TOPL.

[37]  David Leon,et al.  Dex: a semantic-graph differencing tool for studying changes in large code bases , 2004, 20th IEEE International Conference on Software Maintenance, 2004. Proceedings..

[38]  Suzette Person,et al.  Regression Verification Using Impact Summaries , 2013, SPIN.

[39]  Chao Wang,et al.  Peephole Partial Order Reduction , 2008, TACAS.

[40]  Jerod W. Wilkerson.  A software change impact analysis taxonomy , 2012, 2012 28th IEEE International Conference on Software Maintenance (ICSM).

[41]  Gregg Rothermel,et al.  SimRT: an automated framework to support regression testing for data races , 2014, ICSE.

[42]  Chao Wang,et al.  BEST: A symbolic testing tool for predicting multi-threaded program failures , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[43]  Gul Agha,et al.  Scalable Automated Methods for Dynamic Program Analysis , 2006 .

[44]  Benjamin Livshits,et al.  Context-sensitive program analysis as database queries , 2005, PODS.

[45]  Frank Tip,et al.  Associating synchronization constraints with data in an object-oriented language , 2006, POPL '06.

[46]  Vikram S. Adve,et al.  The LLVM Compiler Framework and Infrastructure Tutorial , 2004, LCPC.

[47]  Swarat Chaudhuri,et al.  Symbolic pruning of concurrent program executions , 2009, ESEC/FSE '09.