What would other programmers do: suggesting solutions to error messages

Interpreting compiler errors and exception messages is challenging for novice programmers. Presenting examples of how other programmers have corrected similar errors may help novices understand and correct such errors. This paper introduces HelpMeOut, a social recommender system that aids the debugging of error messages by suggesting solutions that peers have applied in the past. HelpMeOut comprises IDE instrumentation to collect examples of code changes that fix errors; a central database that stores fix reports from many users; and a suggestion interface that, given an error, queries the database for a list of relevant fixes and presents these to the programmer. We report on implementations of this architecture for two programming languages. An evaluation with novice programmers found that the technique can suggest useful fixes for 47% of errors after 39 person-hours of programming in an instrumented environment.

[1]  B. Myers.  Debugging Reinvented: Asking and Answering Why and Why Not Questions about Program Behavior , 2008 .

[2]  Michael I. Jordan,et al.  Scalable statistical bug isolation , 2005, PLDI '05.

[3]  James Fogarty,et al.  Assieme: finding and leveraging implicit references in a web search interface for programmers , 2007, UIST '07.

[4]  Scott R. Klemmer,et al.  Iterative design and evaluation of an event architecture for pen-and-paper interfaces , 2008, UIST '08.

[5]  Benjamin E. Birnbaum,et al.  Achieving flexibility in direct-manipulation programming environments by relaxing the edit-time grammar , 2005, 2005 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC'05).

[6]  Tovi Grossman,et al.  CommunityCommands: command recommendations for software applications , 2009, UIST '09.

[7]  Bonnie A. Nardi,et al.  A Small Matter of Programming: Perspectives on End User Computing , 1993 .

[8]  Vladimir I. Levenshtein,et al.  Binary codes capable of correcting deletions, insertions, and reversals , 1965 .

[9]  Takeo Igarashi,et al.  Generating photo manipulation tutorials by demonstration , 2009, ACM Trans. Graph..

[10]  Sue Fitzgerald,et al.  Debugging: finding, fixing and flailing, a multi-institutional study of novice debuggers , 2008, Comput. Sci. Educ..

[11]  Alfred Kobsa.  User Modeling and User-Adapted Interaction , 2005, User Modeling and User-Adapted Interaction.

[12]  Bertrand Meyer,et al.  Compiler error messages: what can help novices? , 2008, SIGCSE '08.

[13]  Philip J. Guo,et al.  Two studies of opportunistic programming: interleaving web foraging, learning, and writing code , 2009, CHI.

[14]  Randall B. Smith,et al.  From Kansas to Oz: collaborative debugging when a shared world breaks , 1997, CACM.

[15]  K. Holyoak,et al.  Analogical problem solving , 1980, Cognitive Psychology.

[16]  Clinton L. Jeffery,et al.  Generating LR syntax error messages from examples , 2003, TOPL.

[17]  Paul Mulholland,et al.  Fostering debugging communities on the Web , 1997, CACM.

[18]  Bonnie A. Nardi,et al.  A small matter of programming , 1993 .

[19]  Philip J. Guo,et al.  Writing Code to Prototype, Ideate, and Discover , 2009, IEEE Software.

[20]  Sue Fitzgerald,et al.  Debugging: a review of the literature from an educational perspective , 2008, Comput. Sci. Educ..

[21]  Paul Heckel,et al.  A technique for isolating differences between files , 1978, CACM.

[22]  Jurriaan Hage,et al.  Improving type error messages for generic java , 2009, PEPM '09.

[23]  K. Holyoak,et al.  Schema induction and analogical transfer , 1983, Cognitive Psychology.

[24]  Benjamin Livshits,et al.  DynaMine: finding common error patterns by mining software revision histories , 2005, ESEC/FSE-13.

[25]  David F. Redmiles,et al.  Extracting usability information from user interface events , 2000, CSUR.

[26]  Li-Te Cheng,et al.  Building Collaboration into IDEs , 2003, ACM Queue.

[27]  Sunghun Kim,et al.  Memories of bug fixes , 2006, SIGSOFT '06/FSE-14.

[28]  Philip J. Guo,et al.  Opportunistic Programming: Writing Code to Prototype, Ideate, and Discover end-us e r soft war e e ngine e ring , 2009 .

[29]  Zhendong Su,et al.  DECKARD: Scalable and Accurate Tree-Based Detection of Code Clones , 2007, 29th International Conference on Software Engineering (ICSE'07).

[30]  Frank Linton,et al.  Recommender Systems for Learning: Building User and Expert Models through Long-Term Observation of Application Use , 2000, User Modeling and User-Adapted Interaction.

[31]  Thomas Reps,et al.  Programming Techniques and Data Structures , 1981 .

[32]  Matthew Kay,et al.  Ingimp: introducing instrumentation to an end-user open source application , 2008, CHI.

[33]  Scott R. Klemmer,et al.  Example-centric programming: integrating web search into the development environment , 2010, CHI.

[34]  Brad A. Myers,et al.  Improving API documentation using API usage information , 2009, 2009 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[35]  Daniel Shiffman.  Learning Processing: A Beginner's Guide to Programming Images, Animation, and Interaction , 2008 .