Adaptive bug isolation

Statistical debugging uses lightweight instrumentation and statistical models to identify program behaviors that are strongly predictive of failure. However, most software is mostly correct; nearly all monitored behaviors are poor predictors of failure. We propose an adaptive monitoring strategy that mitigates the overhead associated with monitoring poor failure predictors. We begin by monitoring a small portion of the program, then automatically refine instrumentation over time to zero in on bugs. We formulate this approach as a search on the control-dependence graph of the program. We present and evaluate various heuristics that can be used for this search. We also discuss the construction of a binary instrumentor for incorporating the feedback loop into post-deployment monitoring. Performance measurements show that adaptive bug isolation yields an average performance overhead of 1% for a class of large applications, as opposed to 87% for realistic sampling-based instrumentation and 300% for complete binary instrumentation.

[1]  Thomas J. Ostrand,et al.  Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria , 1994, Proceedings of 16th International Conference on Software Engineering.

[2]  Matthew Arnold,et al.  A framework for reducing the cost of instrumented code , 2001, PLDI '01.

[3]  Alessandro Orso,et al.  Techniques for Classifying Executions of Deployed Software to Support Software Engineering Tasks , 2007, IEEE Transactions on Software Engineering.

[4]  Joe D. Warren,et al.  The program dependence graph and its use in optimization , 1987, TOPL.

[5]  Ben Liblit.  Cooperative Bug Isolation (Winning Thesis of the 2005 ACM Doctoral Dissertation Competition) , 2007, Lecture Notes in Computer Science.

[6]  Alessandro Orso,et al.  Gamma system: continuous evolution of software after deployment , 2002, ISSTA '02.

[7]  Benjamin Livshits,et al.  AjaxScope: A Platform for Remotely Monitoring the Client-Side Behavior of Web 2.0 Applications , 2010, ACM Trans. Web.

[8]  Alessandro Orso,et al.  GAMMATELLA: visualization of program-execution data for deployed software , 2004, Proceedings. 26th International Conference on Software Engineering.

[9]  Matthew B. Dwyer,et al.  Adaptive Online Program Analysis , 2007, 29th International Conference on Software Engineering (ICSE'07).

[10]  Zhendong Su,et al.  Context-aware statistical debugging: from bug predictors to faulty control flow paths , 2007, ASE.

[11]  M.L. Soffa,et al.  Demand-driven structural testing with dynamic instrumentation , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[12]  Steven P. Reiss,et al.  Fault localization with nearest neighbor queries , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[13]  Yuanyuan Zhou,et al.  Rx: treating bugs as allergies---a safe method to survive software failures , 2005, SOSP '05.

[14]  John T. Stasko,et al.  Visualization of test information to assist fault localization , 2002, ICSE '02.

[15]  James R. Larus,et al.  Efficient path profiling , 1996, Proceedings of the 29th Annual IEEE/ACM International Symposium on Microarchitecture. MICRO 29.

[16]  Zhenkai Liang,et al.  Fast and automated generation of attack signatures: a basis for building self-protecting servers , 2005, CCS '05.

[17]  Xiaojin Zhu,et al.  Statistical Debugging Using Latent Topic Models , 2007, ECML.

[18]  Mary Jean Harrold,et al.  Empirical evaluation of the tarantula automatic fault-localization technique , 2005, ASE.

[19]  Jeffrey K. Hollingsworth,et al.  An API for Runtime Code Patching , 2000, Int. J. High Perform. Comput. Appl..

[20]  Thomas W. Reps,et al.  CodeSurfer/x86-A Platform for Analyzing x86 Executables , 2005, CC.

[21]  Barton P. Miller,et al.  On-line automated performance diagnosis on thousands of processes , 2006, PPoPP '06.

[22]  Trishul M. Chilimbi,et al.  HOLMES: Effective statistical debugging via efficient path profiling , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[23]  Helen J. Wang,et al.  Shield: vulnerability-driven network filters for preventing known vulnerability exploits , 2004, SIGCOMM.

[24]  Jeffrey K. Hollingsworth,et al.  Efficient online computation of statement coverage , 2005, J. Syst. Softw..

[25]  Michael I. Jordan,et al.  Statistical debugging: simultaneous identification of multiple bugs , 2006, ICML.

[26]  Matthias Hauswirth,et al.  Low-overhead memory leak detection using adaptive statistical profiling , 2004, ASPLOS XI.

[27]  Michael I. Jordan,et al.  Bug isolation via remote program sampling , 2003, PLDI.

[28]  Zhenkai Liang,et al.  Automatic generation of buffer overflow attack signatures: an approach based on program behavior models , 2005, 21st Annual Computer Security Applications Conference (ACSAC'05).

[29]  Helen J. Wang,et al.  BrowserShield: vulnerability-driven filtering of dynamic HTML , 2006, OSDI '06.

[30]  William G. Griswold,et al.  27th International Conference on Software Engineering (ICSE 2005), 15-21 May 2005, St. Louis, Missouri, USA , 2005, ICSE.

[31]  Barton P. Miller,et al.  The Paradyn Parallel Performance Measurement Tool , 1995, Computer.

[32]  Benjamin Livshits,et al.  AjaxScope: a platform for remotely monitoring the client-side behavior of web 2.0 applications , 2007, TWEB.

[33]  Richard M. Stallman,et al.  Using the GNU Compiler Collection , 2010 .

[34]  Susan Horwitz,et al.  Better Debugging via Output Tracing and Callstack-Sensitive Slicing , 2010, IEEE Transactions on Software Engineering.

[35]  Chao Liu,et al.  SOBER: statistical model-based bug localization , 2005, ESEC/FSE-13.

[36]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[37]  Sebastian G. Elbaum,et al.  Reducing coverage collection overhead with disposable instrumentation , 2004, 15th International Symposium on Software Reliability Engineering.

[38]  Michael I. Jordan,et al.  Scalable statistical bug isolation , 2005, PLDI '05.

[39]  Alex Aiken,et al.  Cooperative Bug Isolation , 2007 .

[40]  Thomas W. Reps,et al.  The use of program dependence graphs in software engineering , 1992, International Conference on Software Engineering.

[41]  James Newsome,et al.  Dynamic Taint Analysis for Automatic Detection, Analysis, and SignatureGeneration of Exploits on Commodity Software , 2005, NDSS.

[42]  H. Cleve,et al.  Locating causes of program failures , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..