Using Hypotheses as a Debugging Aid

As developers debug, developers formulate hypotheses about the cause of the defect and gather evidence to test these hypotheses. To better understand the role of hypotheses in debugging, we conducted two studies. In a preliminary study, we found that, even with the benefit of modern internet resources, incorrect hypotheses can cause developers to investigate irrelevant information and block progress. We then conducted a controlled experiment where 20 developers debugged and recorded their hypotheses. We found that developers have few hypotheses, two per defect. Having a correct hypothesis early strongly predicted later success. We also studied the impact of two debugging aids: fault locations and potential hypotheses. Offering fault locations did not help developers formulate more correct hypotheses or debug more successfully. In contrast, offering potential hypotheses made developers six times more likely to succeed. These results demonstrate the potential of future debugging tools that enable finding and sharing relevant hypotheses.

[1]  Robert Hirschfeld,et al.  Studying the advancement in debugging practice of professional software developers , 2014, 2014 IEEE International Symposium on Software Reliability Engineering Workshops.

[2]  A. J. Ko.  Debugging by asking questions about program output , 2006, ICSE '06.

[3]  Thomas D. LaToza,et al.  Maintaining mental models: a study of developer work habits , 2006, ICSE.

[4]  Eugene H. Spafford,et al.  Critical slicing for software fault localization , 1996, ISSTA '96.

[5]  Thomas D. LaToza,et al.  Supporting Software Engineering Research and Education by Annotating Public Videos of Developers Programming , 2019, 2019 IEEE/ACM 12th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE).

[6]  Scott R. Klemmer,et al.  What would other programmers do: suggesting solutions to error messages , 2010, CHI.

[7]  Anneliese Amschler Andrews,et al.  On the role of hypotheses during opportunistic understanding while porting large scale code , 1996, WPC '96. 4th Workshop on Program Comprehension.

[8]  Rachel K. E. Bellamy,et al.  How Programmers Debug, Revisited: An Information Foraging Theory Perspective , 2013, IEEE Transactions on Software Engineering.

[9]  Stanley Letovsky,et al.  Cognitive processes in program comprehension , 1986, J. Syst. Softw..

[10]  Iris Vessey,et al.  Expertise in Debugging Computer Programs: A Process Analysis , 1984, Int. J. Man Mach. Stud..

[11]  Alessandro Bozzon,et al.  Asking the right question in collaborative q&a systems , 2014, HT.

[12]  Xiangyu Zhang,et al.  Pruning dynamic slices with confidence , 2006, PLDI '06.

[13]  Lena Mamykina,et al.  Design lessons from the fastest q&a site in the west , 2011, CHI.

[14]  Rainer Koschke,et al.  How do professional developers comprehend software? , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[15]  Gail C. Murphy,et al.  Asking and Answering Questions during a Programming Change Task , 2008, IEEE Transactions on Software Engineering.

[16]  Andreas Zeller,et al.  Why Programs Fail: A Guide to Systematic Debugging , 2005 .

[17]  Chanchal Kumar Roy,et al.  Answering questions about unanswered questions of Stack Overflow , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[18]  Robert DeLine,et al.  Information Needs in Collocated Software Development Teams , 2007, 29th International Conference on Software Engineering (ICSE'07).

[19]  Stephan Diehl,et al.  Towards a theory of software development expertise , 2018, ESEC/SIGSOFT FSE.

[20]  Brad A. Myers,et al.  An Exploratory Study of How Developers Seek, Relate, and Collect Relevant Information during Software Maintenance Tasks , 2006, IEEE Transactions on Software Engineering.

[21]  Rishabh Singh,et al.  NoFAQ: synthesizing command repairs from examples , 2016, ESEC/SIGSOFT FSE.

[22]  Elliot Soloway,et al.  Mental models and software maintenance , 1986, J. Syst. Softw..

[23]  Janice Singer,et al.  Debugging Revisited: Toward Understanding the Debugging Needs of Contemporary Software Developers , 2013, 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement.

[24]  Xiangyu Zhang,et al.  Precise dynamic slicing algorithms , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[25]  Claire Le Goues,et al.  A Qualitative Study on Framework Debugging , 2019, 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[26]  Premkumar T. Devanbu,et al.  How social Q&A sites are changing knowledge sharing in open source software communities , 2014, CSCW.

[27]  Christoph Treude,et al.  How do programmers ask and answer questions on the web?: NIER track , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[28]  Alessandro Orso,et al.  Are automated debugging techniques actually helping programmers? , 2011, ISSTA '11.

[29]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[30]  David W. Binkley,et al.  Program slicing , 2008, 2008 Frontiers of Software Maintenance.

[31]  Andreas Zeller,et al.  Where is the bug and how is it fixed? an experiment with practitioners , 2017, ESEC/SIGSOFT FSE.

[32]  Thomas D. LaToza,et al.  An Exploratory Study of Live-Streamed Programming , 2019, 2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[33]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[34]  Brad A. Myers,et al.  Debugging reinvented , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[35]  Ruven E. Brooks,et al.  Towards a Theory of the Comprehension of Computer Programs , 1983, Int. J. Man Mach. Stud..

[36]  Thomas D. LaToza,et al.  Developers ask reachability questions , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.