How Developers Visualize Compiler Messages: A Foundational Approach to Notification Construction

Self-explanation is one cognitive strategy through which developers comprehend error notifications. Self-explanation, when left solely to developers, can result in a significant loss of productivity because humans are imperfect and bounded in their cognitive abilities. We argue that modern IDEs offer limited visual affordances for aiding developers with self-explanation, because compilers do not reveal their reasoning about the causes of errors to the developer. The contribution of our paper is a foundational set of visual annotations that aid developers in better comprehending error messages when compilers expose their internal reasoning. We demonstrate through a user study of 28 undergraduate Software Engineering students that our annotations align with the way in which developers self-explain error notifications. We show that these annotations allow developers to give significantly better self-explanations when compared against today's dominant visualization paradigm, and that better self-explanations yield better mental models of notifications. The results of our work suggest that the diagrammatic techniques developers use to explain problems can serve as an effective foundation for how IDEs should visually communicate to developers.

[1]  Martin Erwig,et al.  Let's hear both sides: On combining type-error reporting tools , 2014, 2014 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[2]  Eliezer Kantorowitz,et al.  Automatic generation of useful syntax error messages , 1986, Softw. Pract. Exp..

[3]  Jakob Nielsen,et al.  Heuristic evaluation of user interfaces , 1990, CHI '90.

[4]  Jurriaan Hage,et al.  Improving type error messages for generic java , 2009, PEPM '09.

[5]  Clinton L. Jeffery,et al.  Generating LR syntax error messages from examples , 2003, TOPL.

[6]  Eric Roberts,et al.  An overview of MiniJava , 2001, SIGCSE '01.

[7]  Andrew Luxton-Reilly,et al.  Enhancing syntax error messages appears ineffectual , 2014, ITiCSE '14.

[8]  V. Javier Traver,et al.  On Compiler Error Messages: What They Say and What They Mean , 2010, Adv. Hum. Comput. Interact..

[9]  Bertrand Meyer,et al.  Compiler error messages: what can help novices? , 2008, SIGCSE '08.

[10]  Ben Shneiderman,et al.  Measuring Computer Program Quality and Comprehension , 1977, Int. J. Man Mach. Stud..

[11]  Shaaron Ainsworth,et al.  The effects of self-explaining when learning with text or diagrams , 2003, Cogn. Sci..

[12]  José Nelson Amaral,et al.  Syntax errors just aren't natural: improving error reporting with language models , 2014, MSR 2014.

[13]  Brad A. Myers,et al.  A framework and methodology for studying the causes of software errors in programming systems , 2005, J. Vis. Lang. Comput..

[14]  Scott R. Klemmer,et al.  What would other programmers do: suggesting solutions to error messages , 2010, CHI.

[15]  Chris Parnin,et al.  Subvocalization - Toward Hearing the Inner Thoughts of Developers , 2011, 2011 IEEE 19th International Conference on Program Comprehension.

[16]  Marian Petre,et al.  Usability Analysis of Visual Programming Environments: A 'Cognitive Dimensions' Framework , 1996, J. Vis. Lang. Comput..

[17]  Anind K. Dey,et al.  Why and why not explanations improve the intelligibility of context-aware intelligent systems , 2009, CHI.