Studying the advancement in debugging practice of professional software developers

AbstractIn 1997, Henry Lieberman stated that debugging is the dirty little secret of computer science. Since then, several promising debugging technologies have been developed such as back-in-time debuggers and automatic fault localization methods. However, the last study about the state-of-the-art in debugging is still more than 15 years old and so it is not clear whether these new approaches have been applied in practice or not. For that reason, we investigate the current state of debugging in a comprehensive study. First, we review the available literature and learn about current approaches and study results. Second, we observe several professional developers while debugging and interview them about their experiences. Third, we create a questionnaire that serves as the basis for a larger online debugging survey. Based on these results, we present new insights into debugging practice that help to suggest new directions for future research.

[1]  Mary Jean Harrold,et al.  Debugging in Parallel , 2007, ISSTA '07.

[2]  Chao Liu,et al.  SOBER: statistical model-based bug localization , 2005, ESEC/FSE-13.

[3]  Holger Keding,et al.  The Developer's Guide to Debugging , 2008 .

[4]  Koen De Bosschere,et al.  Proceedings of the Fifth International Workshop on Automated Debugging (AADEBUG 2003) , 2003 .

[5]  Brian W. Kernighan,et al.  Elements of Programming Style , 1974 .

[6]  H. Cleve,et al.  Locating causes of program failures , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[7]  Frank Tip,et al.  Practical fault localization for dynamic web applications , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[8]  Zhendong Su,et al.  Context-aware statistical debugging: from bug predictors to faulty control flow paths , 2007, ASE.

[9]  Rui Abreu,et al.  Zoltar: A Toolset for Automatic Fault Localization , 2009, 2009 IEEE/ACM International Conference on Automated Software Engineering.

[10]  Gary Lewandowski,et al.  Debugging: the good, the bad, and the quirky -- a qualitative analysis of novices' strategies , 2008, SIGCSE '08.

[11]  K. Pearson.  VII. Note on regression and inheritance in the case of two parents , 1895, Proceedings of the Royal Society of London.

[12]  Steven P. Reiss,et al.  Fault localization with nearest neighbor queries , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[13]  John D. Gould,et al.  An Exploratory Study of Computer Program Debugging1 , 1974 .

[14]  Andreas Zeller,et al.  Lightweight Defect Localization for Java , 2005, ECOOP.

[15]  Ben Liblit,et al.  Adaptive bug isolation , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[16]  Bil Lewis,et al.  Debugging Backwards in Time , 2003, ArXiv.

[17]  Joseph Robert Horgan,et al.  Fault localization using execution slices and dataflow tests , 1995, Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'95.

[18]  Richard W. Vuduc,et al.  Falcon: fault localization in concurrent programs , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[19]  Andy Podgurski,et al.  Causal inference for statistical fault localization , 2010, ISSTA '10.

[20]  John A. Gould,et al.  Some Psychological Evidence on How People Debug Computer Programs , 1975, Int. J. Man Mach. Stud..

[21]  Rui Abreu,et al.  Zoltar: a spectrum-based fault localization tool , 2009, SINTER '09.

[22]  Jim Clark.  A PostScript® tutorial: tutorial presentation , 2009 .

[23]  K. Pearson.  On the Criterion that a Given System of Deviations from the Probable in the Case of a Correlated System of Variables is Such that it Can be Reasonably Supposed to have Arisen from Random Sampling , 1900 .

[24]  Scott James,et al.  Why Sally and Joey can't debug: next generation tools and the perils they pose , 2008 .

[25]  Andreas Zeller,et al.  Why Programs Fail: A Guide to Systematic Debugging , 2005 .

[26]  Thomas D. LaToza,et al.  Developers ask reachability questions , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[27]  David J. Agans.  Debugging: The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems , 2002 .

[28]  Robert Charles Metzger.  Debugging by Thinking: A Multidisciplinary Approach , 2003 .

[29]  Brent Hailpern,et al.  Software debugging, testing, and verification , 2002, IBM Syst. J..

[30]  Xiangyu Zhang,et al.  Locating faulty code using failure-inducing chops , 2005, ASE.

[31]  A. Zeller.  Isolating cause-effect chains from computer programs , 2002, SIGSOFT '02/FSE-10.

[32]  Mark Weiser,et al.  Programmers use slices when debugging , 1982, CACM.

[33]  Michael I. Jordan,et al.  Scalable statistical bug isolation , 2005, PLDI '05.

[34]  Michael Perscheid,et al.  Test-driven Fault Navigation for Debugging Reproducible Failures , 2012, Inf. Media Technol..

[35]  Karl Pearson F.R.S..  X. On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling , 2009 .

[36]  Marc Eisenstadt,et al.  My hairiest bug war stories , 1997, CACM.

[37]  Michael C. Loui,et al.  Debugging: from novice to expert , 2004, SIGCSE '04.

[38]  Henry Lieberman,et al.  The Debugging Scandal and What to Do About It (Introduction to the Special Section) , 1997, Commun. ACM.

[39]  Xiangyu Zhang,et al.  Locating faults through automated predicate switching , 2006, ICSE.

[40]  Michael C. Loui,et al.  Debugging: from novice to expert , 2004 .

[41]  Iris Vessey,et al.  Expertise in Debugging Computer Programs: A Process Analysis , 1984, Int. J. Man Mach. Stud..

[42]  Rajiv Gupta,et al.  Fault localization using value replacement , 2008, ISSTA '08.

[43]  Brian Hanks,et al.  Successful and unsuccessful problem solving approaches of novice programmers , 2009, SIGCSE '09.

[44]  Raimondas Lencevicius,et al.  On-the-fly query-based debugging with examples , 2000, AADEBUG.