BugRedux: Reproducing field failures for in-house debugging

A recent survey conducted among developers of the Apache, Eclipse, and Mozilla projects showed that the ability to recreate field failures is considered of fundamental importance when investigating bug reports. Unfortunately, the information typically contained in a bug report, such as memory dumps or call stacks, is usually insufficient for recreating the problem. Even more advanced approaches for gathering field data and help in-house debugging tend to collect either too little information, and be ineffective, or too much information, and be inefficient. To address these issues, we present BugRedux, a novel general approach for in-house debugging of field failures. BugRedux aims to synthesize, using execution data collected in the field, executions that mimic the observed field failures. We define several instances of BugRedux that collect different types of execution data and perform, through an empirical study, a cost-benefit analysis of the approach and its variations. In the study, we apply BugRedux to 16 failures of 14 real-world programs. Our results are promising in that they show that it is possible to synthesize in-house executions that reproduce failures observed in the field using a suitable set of execution data.

[1]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[2]  Alfred V. Aho,et al.  Compilers: Principles, Techniques, and Tools , 1986, Addison-Wesley series in computer science / World student series edition.

[3]  Michal Young,et al.  Residual test coverage monitoring , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[4]  David F. Redmiles,et al.  Extracting usability information from user interface events , 2000, CSUR.

[5]  Greg Nelson,et al.  Extended static checking for Java , 2002, PLDI '02.

[6]  Andreas Zeller,et al.  Simplifying and Isolating Failure-Inducing Input , 2002, IEEE Trans. Software Eng..

[7]  Sarfraz Khurshid,et al.  Test input generation with java PathFinder , 2004, ISSTA '04.

[8]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[9]  Koushik Sen,et al.  DART: directed automated random testing , 2005, PLDI '05.

[10]  Michael I. Jordan,et al.  Scalable statistical bug isolation , 2005, PLDI '05.

[11]  Yuanyuan Zhou,et al.  BugBench: Benchmarks for Evaluating Bug Detection Tools , 2005 .

[12]  Madeline Diep,et al.  Profiling deployed software: assessing strategies and testing opportunities , 2005, IEEE Transactions on Software Engineering.

[13]  Alfred V. Aho,et al.  Compilers: Principles, Techniques, and Tools (2nd Edition) , 2006 .

[14]  Alex Aiken,et al.  Cooperative Bug Isolation , 2007 .

[15]  Alessandro Orso,et al.  A Technique for Enabling and Supporting Debugging of Field Failures , 2007, 29th International Conference on Software Engineering (ICSE'07).

[16]  Zhendong Su,et al.  Context-aware statistical debugging: from bug predictors to faulty control flow paths , 2007, ASE.

[17]  Michael D. Ernst,et al.  Randoop: feedback-directed random testing for Java , 2007, OOPSLA '07.

[18]  Miguel Castro,et al.  Better bug reporting with better privacy , 2008, ASPLOS 2008.

[19]  Shuvendu K. Lahiri,et al.  Finding errors in .net with feedback-directed random testing , 2008, ISSTA '08.

[20]  Patrice Godefroid,et al.  Automated Whitebox Fuzz Testing , 2008, NDSS.

[21]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[22]  Michael D. Ernst,et al.  ReCrash: Making Software Failures Reproducible by Preserving Object States , 2008, ECOOP.

[23]  Yuanyuan Zhou,et al.  PRES: probabilistic replay with execution sketching on multiprocessors , 2009, SOSP '09.

[24]  Sriram K. Rajamani,et al.  DebugAdvisor: a recommender system for debugging , 2009, ESEC/FSE '09.

[25]  Manu Sridharan,et al.  Snugglebug: a powerful approach to weakest preconditions , 2009, PLDI '09.

[26]  Trishul M. Chilimbi,et al.  HOLMES: Effective statistical debugging via efficient path profiling , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[27]  Alessandro Orso,et al.  Camouflage: Automated Sanitization of Field Data , 2009 .

[28]  Mangala Gowri Nanda,et al.  Accurate Interprocedural Null-Dereference Analysis for Java , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[29]  Mark Harman,et al.  A Theoretical and Empirical Study of Search-Based Testing: Local, Global, and Hybrid Search , 2010, IEEE Transactions on Software Engineering.

[30]  George Candea,et al.  Execution synthesis: a technique for automated software debugging , 2010, EuroSys '10.

[31]  Thomas Zimmermann,et al.  What Makes a Good Bug Report? , 2008, IEEE Transactions on Software Engineering.

[32]  Ding Yuan,et al.  SherLog: error diagnosis by connecting clues from run-time logs , 2010, ASPLOS XV.

[33]  David Brumley,et al.  AEG: Automatic Exploit Generation , 2011, NDSS.

[34]  Alessandro Orso,et al.  Camouflage: automated anonymization of field data , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[35]  Alessandro Orso,et al.  Are automated debugging techniques actually helping programmers? , 2011, ISSTA '11.

[36]  Ricardo Bianchini,et al.  Striking a new balance between program instrumentation and debugging time , 2011, EuroSys '11.

[37]  Ding Yuan,et al.  Improving Software Diagnosability via Log Enhancement , 2012, TOCS.

[38]  David Brumley,et al.  Automatic exploit generation , 2014, CACM.