Garbage in, garbage out? An empirical look at oracle mistakes by end-user programmers

End-user programmers, because they are human, make mistakes. However, past research has not considered how visual end-user debugging devices could be designed to ameliorate the effects of mistakes. This paper empirically examines oracle mistakes - mistakes users make about which values are right and which are wrong - to reveal differences in how different types of oracle mistakes impact the quality of visual feedback about bugs. We then consider the implications of these empirical results for designers of end-user software engineering environments.

[1]  Margaret M. Burnett,et al.  End-user software visualizations for fault localization , 2003, SoftVis '03.

[2]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[3]  Jorma Sajaniemi.  Modeling Spreadsheet Audit: A Rigorous Approach to Automatic Visualization , 2000, J. Vis. Lang. Comput..

[4]  G. Rothermel,et al.  An empirical study of fault localization for end-user programmers , 2005, ICSE.

[5]  Raymond R. Panko,et al.  What we know about spreadsheet errors , 1998 .

[6]  Bay-Wei Chang,et al.  Fluid visualization of spreadsheet structures , 1998, Proceedings. 1998 IEEE Symposium on Visual Languages (Cat. No.98TB100254).

[7]  Ellis Horowitz,et al.  Software Cost Estimation with COCOMO II , 2000 .

[8]  J.Steve Davis.  Tools for spreadsheet auditing , 1996, Int. J. Hum. Comput. Stud..

[9]  Gregg Rothermel,et al.  End-user software engineering with assertions in the spreadsheet paradigm , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[10]  Dennis F. Galletta,et al.  An experimental study of spreadsheet presentation and error detection , 1996, Proceedings of HICSS-29: 29th Hawaii International Conference on System Sciences.

[11]  Margaret M. Burnett,et al.  Rewarding "Good" Behavior: End-User Debugging and Rewards , 2004, 2004 IEEE Symposium on Visual Languages - Human Centric Computing.

[12]  김종덕,et al.  Interactive. , 1996, Nursing older people.

[13]  Brad A. Myers,et al.  Development and evaluation of a model of programming errors , 2003, IEEE Symposium on Human Centric Computing Languages and Environments, 2003. Proceedings. 2003.

[14]  Dennis F. Galletta,et al.  Spreadsheet Presentation and Error Detection: An Experimental Study , 1997, J. Manag. Inf. Syst..

[15]  Margaret M. Burnett,et al.  Journal of Visual Languages & Computing Interactive, Visual Fault Localization Support for End-user Programmers This Paper Updates and Extends Earlier Work That Appeared In , 2022 .

[16]  Mary Shaw,et al.  Estimating the numbers of end users and end user programmers , 2005, 2005 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC'05).

[17]  Gregg Rothermel,et al.  A methodology for testing spreadsheets , 2001, TSEM.

[18]  Margaret M. Burnett,et al.  Effectiveness of end-user debugging software features: are there gender issues? , 2005, CHI.

[19]  Henry Lieberman,et al.  Supporting User Hypotheses in Problem Diagnosis on the Web and Elsewhere , 2004 .

[20]  Shriram Krishnamurthi,et al.  A type system for statically detecting spreadsheet errors , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[21]  Gregg Rothermel,et al.  End-user software engineering , 2004, Commun. ACM.

[22]  Martin Erwig,et al.  Header and Unit Inference for Spreadsheets Through Spatial Analyses , 2004, 2004 IEEE Symposium on Visual Languages - Human Centric Computing.

[23]  Eugene Creswick,et al.  Strategies and behaviors of end-user programmers with interactive fault localization , 2003, IEEE Symposium on Human Centric Computing Languages and Environments, 2003. Proceedings. 2003.

[24]  Margaret M. Burnett,et al.  Forms/3: A first-order visual language to explore the boundaries of the spreadsheet paradigm , 2001, Journal of Functional Programming.

[25]  Roland Mittermeir,et al.  Spreadsheet Debugging , 2003, ArXiv.

[26]  Elaine J. Weyuker,et al.  On Testing Non-Testable Programs , 1982, Comput. J..

[27]  Margaret Burnett,et al.  Interactive fault localization techniques to empower the debugging efforts of end-user programmers , 2004 .

[28]  Brad A. Myers,et al.  Designing the whyline: a debugging interface for asking questions about program behavior , 2004, CHI.