On the use of delta debugging to reduce recordings and facilitate debugging of web applications

Recording the sequence of events that lead to a failure of a web application can be an effective aid for debugging. Nevertheless, a recording of an event sequence may include many events that are not related to a failure, and this may render debugging more difficult. To address this problem, we have adapted Delta Debugging to function on recordings of web applications, in a manner that lets it identify and discard portions of those recordings that do not influence the occurrence of a failure. We present the results of three empirical studies that show that (1) recording reduction can achieve significant reductions in recording size and replay time on actual web applications obtained from developer forums, (2) reduced recordings do in fact help programmers locate faults significantly more efficiently as, and no less effectively than non-reduced recordings, and (3) recording reduction produces even greater reductions on larger, more complex applications.

[1]  Andreas Zeller,et al.  Efficient unit test case minimization , 2007, ASE '07.

[2]  Shriram Krishnamurthi,et al.  Automated Fault Localization Using Potential Invariants , 2003, ArXiv.

[3]  Rob Miller,et al.  Sikuli: using GUI screenshots for search and automation , 2009, UIST '09.

[4]  Jan Vitek,et al.  Automated construction of JavaScript benchmarks , 2011, OOPSLA '11.

[5]  Ali Mesbah,et al.  Vejovis: suggesting fixes for JavaScript faults , 2014, ICSE.

[6]  Arie van Deursen,et al.  Invariant-Based Automatic Testing of Modern Web Applications , 2012, IEEE Transactions on Software Engineering.

[7]  Frank Tip,et al.  Finding bugs in dynamic web applications , 2008, ISSTA '08.

[8]  Thomas Zimmermann,et al.  What Makes a Good Bug Report? , 2008, IEEE Transactions on Software Engineering.

[9]  Yannis Smaragdakis,et al.  DSD-Crasher: A hybrid analysis tool for bug finding , 2006, TSEM.

[10]  Huai Liu,et al.  Enhancing adaptive random testing for programs with high dimensional input domains or failure-unrelated parameters , 2008, Software Quality Journal.

[11]  Guy Wiener,et al.  A decentralized approach for programming interactive applications with JavaScript and blockly , 2012, AGERE! 2012.

[12]  Koushik Sen,et al.  Jalangi: a selective record-replay and dynamic analysis framework for JavaScript , 2013, ESEC/FSE 2013.

[13]  Michael D. Ernst,et al.  Interactive record/replay for web application debugging , 2013, UIST.

[14]  A. Zeller Isolating cause-effect chains from computer programs , 2002, SIGSOFT '02/FSE-10.

[15]  Olga Korosteleva Nonparametric Methods in Statistics with SAS Applications , 2013 .

[16]  Andreas Zeller,et al.  Simplifying and Isolating Failure-Inducing Input , 2002, IEEE Trans. Software Eng..

[17]  Galen C. Hunt,et al.  Debugging in the (very) large: ten years of implementation and experience , 2009, SOSP '09.

[18]  Robert H. B. Netzer,et al.  Optimal tracing and incremental reexecution for debugging long-running programs , 1994, PLDI '94.

[19]  Ali Mesbah,et al.  PYTHIA: Generating test cases with oracles for JavaScript applications , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[20]  Alessandro Orso,et al.  A Technique for Enabling and Supporting Debugging of Field Failures , 2007, 29th International Conference on Software Engineering (ICSE'07).

[21]  Gregg Rothermel,et al.  Using assertions to help end-user programmers create dependable web macros , 2008, SIGSOFT '08/FSE-16.

[22]  Shriram Krishnamurthi,et al.  Automated Fault Localization Using Potential Invariants , 2003, ArXiv.

[23]  Ali Mesbah,et al.  AutoFLox: An Automatic Fault Localizer for Client-Side JavaScript , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[24]  Eben M. Haber,et al.  CoScripter: automating & sharing how-to knowledge in the enterprise , 2008, CHI.

[25]  W. Kent Fuchs,et al.  Reversible Debugging Using Program Instrumentation , 2001, IEEE Trans. Software Eng..

[26]  Frank Tip,et al.  Practical fault localization for dynamic web applications , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[27]  Ali Mesbah,et al.  JSART: JavaScript Assertion-Based Regression Testing , 2012, ICWE.

[28]  Willem Visser,et al.  Variably interprocedural program analysis for runtime error detection , 2007, ISSTA '07.

[29]  Martin Burger,et al.  Minimizing reproduction of software failures , 2011, ISSTA '11.

[30]  Xing Zhang,et al.  Feedlack detects missing feedback in web applications , 2011, CHI.

[31]  Mark Christiaens,et al.  A Taxonomy of Execution Replay Systems , 2003 .

[32]  T. H. Tse,et al.  Capturing propagation of infected program states , 2009, ESEC/FSE '09.

[33]  Xiangyu Zhang,et al.  Virtual DOM coverage for effective testing of dynamic web applications , 2014, ISSTA 2014.

[34]  Alex Groce,et al.  Extended program invariants: applications in testing and fault localization , 2012, WODA 2012.

[35]  Michael I. Jordan,et al.  Scalable statistical bug isolation , 2005, PLDI '05.

[36]  Steven P. Reiss,et al.  Fault localization with nearest neighbor queries , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[37]  Satish Narayanasamy,et al.  BugNet: continuously recording program execution for deterministic replay debugging , 2005, 32nd International Symposium on Computer Architecture (ISCA'05).

[38]  Jon Howell,et al.  Mugshot: Deterministic Capture and Replay for JavaScript Applications , 2010, NSDI.

[39]  Alessandro Orso,et al.  Reproducing Field Failures for Programs with Complex Grammar-Based Input , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation.

[40]  Gene Cooperman,et al.  Semi-automated debugging via binary search through a process lifetime , 2013, PLOS '13.

[41]  Samuel T. King,et al.  Debugging Operating Systems with Time-Traveling Virtual Machines (Awarded General Track Best Paper Award!) , 2005, USENIX Annual Technical Conference, General Track.

[42]  T. H. Tse,et al.  Fault Localization with Non-parametric Program Behavior Model , 2008, 2008 The Eighth International Conference on Quality Software.

[43]  Yuriy Brun,et al.  Software Fault Identification via Dynamic Analysis and Machine Learning , 2003 .

[44]  Brad A. Myers,et al.  Extracting and answering why and why not questions about Java program output , 2010, TSEM.

[45]  Alessandro Orso,et al.  Are automated debugging techniques actually helping programmers? , 2011, ISSTA '11.

[46]  Andreas Zeller,et al.  Finding Failure Causes through Automated Testing , 2000, AADEBUG.

[47]  Ali Mesbah,et al.  Efficient JavaScript Mutation Testing , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[48]  Alessandro Orso,et al.  Selective capture and replay of program executions , 2005, WODA '05.

[49]  Yong Lei,et al.  Minimization of randomized unit test cases , 2005, 16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05).

[50]  Jeff Huang,et al.  CLAP: recording local executions to reproduce concurrency failures , 2013, PLDI.

[51]  Silviu Andrica,et al.  WaRR: A tool for high-fidelity web application record and replay , 2011, 2011 IEEE/IFIP 41st International Conference on Dependable Systems & Networks (DSN).

[52]  Xuejun Yang,et al.  Test-case reduction for C compiler bugs , 2012, PLDI.

[53]  Amin Milani Fard,et al.  Leveraging existing tests in automated test generation for web applications , 2014, ASE.

[54]  Srikanth Kandula,et al.  Flashback: A Lightweight Extension for Rollback and Deterministic Replay for Software Debugging , 2004, USENIX Annual Technical Conference, General Track.

[55]  Thomas Zimmermann,et al.  What Makes a Good Bug Report? , 2010, IEEE Trans. Software Eng..

[56]  John T. Stasko,et al.  Visualization of test information to assist fault localization , 2002, ICSE '02.