STAR: Stack Trace Based Automatic Crash Reproduction via Symbolic Execution

Software crash reproduction is the necessary first step for debugging. Unfortunately, crash reproduction is often labor intensive. To automate crash reproduction, many techniques have been proposed including record-replay and post-failure-process approaches. Record-replay approaches can reliably replay recorded crashes, but they incur substantial performance overhead to program executions. Alternatively, post-failure-process approaches analyse crashes only after they have occurred. Therefore they do not incur performance overhead. However, existing post-failure-process approaches still cannot reproduce many crashes in practice because of scalability issues and the object creation challenge. This paper proposes an automatic crash reproduction framework using collected crash stack traces. The proposed approach combines an efficient backward symbolic execution and a novel method sequence composition approach to generate unit test cases that can reproduce the original crashes without incurring additional runtime overhead. Our evaluation study shows that our approach successfully exploited 31 (59.6 percent) of 52 crashes in three open source projects. Among these exploitable crashes, 22 (42.3 percent) are useful reproductions of the original crashes that reveal the crash triggering bugs. A comparison study also demonstrates that our approach can effectively outperform existing crash reproduction approaches.

[1]  David W. Binkley,et al.  Program slicing , 2008, 2008 Frontiers of Software Maintenance.

[2]  Alan J. Hu,et al.  Calysto: scalable and precise extended static checking , 2008, ICSE.

[3]  Edsger W. Dijkstra,et al.  A Discipline of Programming , 1976 .

[4]  Flemming Nielson,et al.  Principles of Program Analysis , 1999, Springer Berlin Heidelberg.

[5]  Nikolai Tillmann,et al.  Pex-White Box Test Generation for .NET , 2008, TAP.

[6]  Manu Sridharan,et al.  Snugglebug: a powerful approach to weakest preconditions , 2009, PLDI '09.

[7]  Mangala Gowri Nanda,et al.  Accurate Interprocedural Null-Dereference Analysis for Java , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[8]  George Candea,et al.  Execution synthesis: a technique for automated software debugging , 2010, EuroSys '10.

[9]  Lars Ole Andersen,et al.  Program Analysis and Specialization for the C Programming Language , 2005 .

[10]  Bertrand Meyer,et al.  On the Effectiveness of Test Extraction without Overhead , 2009, 2009 International Conference on Software Testing Verification and Validation.

[11]  Carl K. Chang,et al.  OCAT: object capture-based automated testing , 2010, ISSTA '10.

[12]  Michael D. Ernst,et al.  Automatic test factoring for java , 2005, ASE '05.

[13]  Nikolai Tillmann,et al.  Demand-Driven Compositional Symbolic Execution , 2008, TACAS.

[14]  Xiangyu Zhang,et al.  Analyzing multicore dumps to facilitate concurrency bug reproduction , 2010, ASPLOS XV.

[15]  Joseph Robert Horgan,et al.  Dynamic program slicing , 1990, PLDI '90.

[16]  Amiram Yehudai,et al.  Regression Test Selection Techniques for Test-Driven Development , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[17]  Dawson R. Engler,et al.  RWset: Attacking Path Explosion in Constraint-Based Test Generation , 2008, TACAS.

[18]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[19]  Nikolai Tillmann,et al.  MSeqGen: object-oriented unit-test generation via mining source code , 2009, ESEC/SIGSOFT FSE.

[20]  John C. Platt,et al.  Finding Similar Failures Using Callstack Similarity , 2008, SysML.

[21]  Matthew B. Dwyer,et al.  Green: reducing, reusing and recycling constraints in program analysis , 2012, SIGSOFT FSE.

[22]  Koushik Sen,et al.  Randomized active atomicity violation detection in concurrent programs , 2008, SIGSOFT '08/FSE-16.

[23]  John McCarthy,et al.  Towards a Mathematical Science of Computation , 1962, IFIP Congress.

[24]  Patrice Godefroid,et al.  Compositional dynamic test generation , 2007, POPL '07.

[25]  Nikolai Tillmann,et al.  Precise identification of problems for structural test generation , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[26]  Michael D. Ernst,et al.  ReCrash: Making Software Failures Reproducible by Preserving Object States , 2008, ECOOP.

[27]  Jianjun Zhao,et al.  A Lightweight and Portable Approach to Making Concurrent Failures Reproducible , 2010, FASE.

[28]  Nikolai Tillmann,et al.  eXpress: guided path exploration for efficient regression test generation , 2011, ISSTA '11.

[29]  Alessandro Orso,et al.  A Technique for Enabling and Supporting Debugging of Field Failures , 2007, 29th International Conference on Software Engineering (ICSE'07).

[30]  Manu Sridharan,et al.  PSE: explaining program failures via postmortem static analysis , 2004, SIGSOFT '04/FSE-12.

[31]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.

[32]  Galen C. Hunt,et al.  Debugging in the (very) large: ten years of implementation and experience , 2009, SOSP '09.

[33]  Marat Boshernitsan,et al.  Predicting Effectiveness of Automatic Testing Tools , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[34]  Xiangyu Zhang,et al.  Z3-str: a z3-based string solver for web application analysis , 2013, ESEC/FSE 2013.

[35]  Zhendong Su,et al.  Synthesizing method sequences for high-coverage testing , 2011, OOPSLA '11.

[36]  Thomas Zimmermann,et al.  What Makes a Good Bug Report? , 2008, IEEE Transactions on Software Engineering.

[37]  Dawson R. Engler,et al.  EXE: automatically generating inputs of death , 2006, CCS '06.

[38]  Yuanyuan Zhou,et al.  PRES: probabilistic replay with execution sketching on multiprocessors , 2009, SOSP '09.

[39]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[40]  Ion Stoica,et al.  ODR: output-deterministic replay for multicore debugging , 2009, SOSP '09.

[41]  Corina S. Pasareanu,et al.  Symbolic execution with mixed concrete-symbolic solving , 2011, ISSTA '11.

[42]  Ning Chen,et al.  Mining Crash Fix Patterns , 2013, ArXiv.

[43]  Nicolás Serrano,et al.  Bugzilla, ITracker, and Other Bug Trackers , 2005, IEEE Softw..

[44]  Ning Chen,et al.  Puzzle-based automatic testing: bringing humans into the loop by solving puzzles , 2012, 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering.

[45]  Satish Narayanasamy,et al.  BugNet: continuously recording program execution for deterministic replay debugging , 2005, 32nd International Symposium on Computer Architecture (ISCA'05).

[46]  Yannis Smaragdakis,et al.  JCrasher: an automatic robustness tester for Java , 2004, Softw. Pract. Exp..

[47]  Bertrand Meyer,et al.  Contract driven development = test driven development - writing test cases , 2007, ESEC-FSE '07.

[48]  John McCarthy,et al.  A basis for a mathematical theory of computation, preliminary report , 1899, IRE-AIEE-ACM '61 (Western).

[49]  Michael D. Ernst,et al.  HAMPI: a solver for string constraints , 2009, ISSTA.

[50]  Michael D. Ernst,et al.  Feedback-Directed Random Test Generation , 2007, 29th International Conference on Software Engineering (ICSE'07).

[51]  Ding Yuan,et al.  Improving Software Diagnosability via Log Enhancement , 2012, TOCS.

[52]  Andreas Zeller,et al.  Reconstructing Core Dumps , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[53]  Alessandro Orso,et al.  Optimizing Constraint Solving to Better Support Symbolic Execution , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[54]  Alessandro Orso,et al.  BugRedux: Reproducing field failures for in-house debugging , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[55]  Ding Yuan,et al.  SherLog: error diagnosis by connecting clues from run-time logs , 2010, ASPLOS XV.

[56]  John Steven,et al.  jRapture: A Capture/Replay tool for observation-based testing , 2000, ISSTA '00.