SymCrash: selective recording for reproducing crashes

Software often crashes despite tremendous effort on software quality assurance. Once developers receive a crash report, they need to reproduce the crash in order to understand the problem and locate the fault. However, limited information from crash reports often makes crash reproduction difficult. Many "capture-and-replay" techniques have been proposed to automatically capture program execution data from the failing code, and help developers replay the crash scenarios based on the captured data. However, such techniques often suffer from heavy overhead and introduce privacy concerns. Recently, methods such as BugRedux were proposed to generate test input that leads to crash through symbolic execution. However, such methods have inherent limitations because they rely on conventional symbolic execution techniques. In this paper, we propose a dynamic symbolic execution method called SymCon, which addresses the limitation of conventional symbolic execution by selecting functions that are hard to be resolved by a constraint solver and using their concrete runtime values to replace the symbols. We then propose SymCrash, a selective recording approach that only instruments and monitors the hard-to-solve functions. SymCrash can generate test input for crashes through SymCon. We have applied our approach to successfully reproduce 13 failures of 6 real-world programs. Our results confirm that the proposed approach is suitable for reproducing crashes, in terms of effectiveness, overhead, and privacy. It also outperforms the related methods.

[1]  Gail E. Kaiser,et al.  Chronicler: Lightweight recording to reproduce field failures , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[2]  Alessandro Orso,et al.  BugRedux: Reproducing field failures for in-house debugging , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[3]  L. D. Moura,et al.  The YICES SMT Solver , 2006 .

[4]  Sriram K. Rajamani,et al.  DebugAdvisor: a recommender system for debugging , 2009, ESEC/FSE '09.

[5]  Mark Russinovich,et al.  Replay for concurrent non-deterministic shared-memory applications , 1996, PLDI '96.

[6]  Dongmei Zhang,et al.  ReBucket: A method for clustering duplicate crash reports based on call stack similarity , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[7]  Nikolai Tillmann,et al.  Pex4Fun: Teaching and learning computer science via social gaming , 2011, 2011 24th IEEE-CS Conference on Software Engineering Education and Training (CSEE&T).

[8]  Archana Ganapathi,et al.  Windows XP Kernel Crash Analysis , 2006, LISA.

[9]  Nikolai Tillmann,et al.  Precise identification of problems for structural test generation , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[10]  Laurie Hendren,et al.  Soot---a java optimization framework , 1999 .

[11]  Corina S. Pasareanu,et al.  JPF-SE: A Symbolic Execution Extension to Java PathFinder , 2007, TACAS.

[12]  Yuanyuan Zhou,et al.  Triage: diagnosing production run failures at the user's site , 2007, SOSP.

[13]  Alessandro Orso,et al.  Selective capture and replay of program executions , 2005, WODA '05.

[14]  James C. Browne,et al.  Model Checking Software via Abstraction of Loop Transitions , 2003, FASE.

[15]  Nikolai Tillmann,et al.  Fitness-guided path exploration in dynamic symbolic execution , 2009, 2009 IEEE/IFIP International Conference on Dependable Systems & Networks.

[16]  Corina S. Pasareanu,et al.  Symbolic Execution with Abstract Subsumption Checking , 2006, SPIN.

[17]  Kenneth L. McMillan Lazy Annotation for Program Testing and Verification , 2010, CAV.

[18]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[19]  Alessandro Orso,et al.  SCARPE: A Technique and Tool for Selective Capture and Replay of Program Executions , 2007, 2007 IEEE International Conference on Software Maintenance.

[20]  Corina S. Pasareanu,et al.  Verification of Java Programs Using Symbolic Execution and Invariant Generation , 2004, SPIN.

[21]  Thomas Zimmermann,et al.  What Makes a Good Bug Report? , 2008, IEEE Transactions on Software Engineering.

[22]  Dawson R. Engler,et al.  EXE: Automatically Generating Inputs of Death , 2008, TSEC.

[23]  P. Saxena,et al.  Modeling Imperative String Operations with Transducers , 2010 .

[24]  Yannis Smaragdakis,et al.  DSD-Crasher: A hybrid analysis tool for bug finding , 2006, TSEM.

[25]  Michael D. Ernst,et al.  Feedback-Directed Random Test Generation , 2007, 29th International Conference on Software Engineering (ICSE'07).

[26]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[27]  Koushik Sen,et al.  DART: directed automated random testing , 2005, PLDI '05.

[28]  Bernd Brügge,et al.  Monitoring user interactions for supporting failure reproduction , 2013, 2013 21st International Conference on Program Comprehension (ICPC).

[29]  Thierry Coupaye,et al.  ASM: a code manipulation tool to implement adaptable systems , 2002 .

[30]  Rongxin Wu,et al.  CrashLocator: locating crashing faults based on crash stacks , 2014, ISSTA 2014.

[31]  Michael D. Ernst,et al.  ReCrashJ: a tool for capturing and reproducing program crashes in deployed applications , 2009, ESEC/SIGSOFT FSE.

[32]  Corina S. Pasareanu,et al.  Symbolic PathFinder: integrating symbolic execution with model checking for Java bytecode analysis , 2013, Automated Software Engineering.

[33]  C. Vaccarezza,et al.  The Star Project , 2014 .

[34]  Michael D. Ernst,et al.  ReCrash: Making Software Failures Reproducible by Preserving Object States , 2008, ECOOP.

[35]  Stephen McCamant,et al.  Loop-extended symbolic execution on binary programs , 2009, ISSTA.

[36]  Nikolai Tillmann,et al.  Pex-White Box Test Generation for .NET , 2008, TAP.

[37]  Andreas Zeller,et al.  Why Programs Fail: A Guide to Systematic Debugging , 2005 .

[38]  Galen C. Hunt,et al.  Debugging in the (very) large: ten years of implementation and experience , 2009, SOSP '09.

[39]  Alessandro Orso,et al.  F3: fault localization for field failures , 2013, ISSTA.

[40]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[41]  Patrice Godefroid,et al.  Automatic partial loop summarization in dynamic test generation , 2011, ISSTA '11.

[42]  Nikolai Tillmann,et al.  Characteristic studies of loop problems for structural test generation via symbolic execution , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[43]  Gideon Redelinghuys,et al.  Symbolic string execution , 2012 .

[44]  C. Csallner,et al.  Check 'n' crash: combining static checking and testing , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[45]  R. Nigel Horspool,et al.  Code hunt: searching for secret code for fun , 2014, SBST 2014.

[46]  George Candea,et al.  Execution synthesis: a technique for automated software debugging , 2010, EuroSys '10.

[47]  Yannis Smaragdakis,et al.  JCrasher: an automatic robustness tester for Java , 2004, Softw. Pract. Exp..

[48]  Foutse Khomh,et al.  Classifying field crash reports for fixing bugs: A case study of Mozilla Firefox , 2011, 2011 27th IEEE International Conference on Software Maintenance (ICSM).

[49]  Klaus Havelund,et al.  Model Checking Programs , 2004, Automated Software Engineering.

[50]  Nachiappan Nagappan,et al.  Crash graphs: An aggregated view of multiple crashes to improve crash triage , 2011, 2011 IEEE/IFIP 41st International Conference on Dependable Systems & Networks (DSN).

[51]  Alexander Aiken,et al.  Effective static race detection for Java , 2006, PLDI '06.