Two Experiments in Reducing Overconfidence in Spreadsheet Development

This paper describes two experiments that examined overconfidence in spreadsheet development. Overconfidence has been seen widely in spreadsheet development and could account for the rarity of testing by end-user spreadsheet developers. The first experiment studied a new way of measuring overconfidence. It demonstrated that overconfidence really is strong among spreadsheet developers. The second experiment attempted to reduce overconfidence by telling subjects in the treatment group the percentage of students who made errors on the task in the past. This warning did reduce overconfidence, and it reduced errors somewhat, although not enough to make spreadsheet development safe.

[1]  Ralph H. Sprague,et al.  Hitting the wall: errors in developing and code inspecting a 'simple' spreadsheet model , 1998, Decis. Support Syst..

[2]  L. Ross,et al.  The overconfidence effect in social prediction. , 1990, Journal of personality and social psychology.

[3]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[4]  Mark P. Sena,et al.  A Picture is Worth $84: An Examination of Consumer Behavior on Ebay Motors , 2004, AMCIS.

[5]  George M. Kasper,et al.  A Theory of Decision Support System Design for User Calibration , 1996, Inf. Syst. Res..

[6]  Jens Rasmussen,et al.  The role of error in organizing behaviour* , 1990 .

[7]  Frank R. Clarke.  Confidence Ratings, Second‐Choice Responses, and Confusion Matrices in Intelligibility Tests , 1960 .

[8]  Carl Martin Allwood,et al.  Error Detection Processes in Statistical Problem Solving , 1984, Cogn. Sci..

[9]  B. Fischhoff,et al.  Training for calibration. , 1980 .

[10]  Briony D. Pulford,et al.  Overconfidence, base rates and outcome positivity/negativity of predicted events , 1996 .

[11]  Michael Fagan.  Design and Code Inspections to Reduce Errors in Program Development , 1976, IBM Syst. J..

[12]  Herbert Solomon,et al.  Two models of group behavior in the solution of eureka-type problems , 1955 .

[13]  William Brescia,et al.  Online Calculator Training in Mathematics and Technology , 2006, J. Cases Inf. Technol..

[14]  Khawaja A. Saeed,et al.  A Meta-Analysis Approach toward the Development of an Integrative Framework for Online Consumer Behavior Research , 2004 .

[15]  Raymond R. Panko,et al.  Are Two Heads Better than One? (At Reducing Errors in Spreadsheet Modeling). , 1997 .

[16]  Steve Clarke,et al.  End-user Computing: Concepts, Methodologies, Tools and Applications , 2008 .

[17]  José Palazzo Moreira de Oliveira,et al.  Modeling Learner's Cognitive Abilities in the Context of a Web-Based Learning Environment , 2008 .

[18]  H. Arkes,et al.  Two methods of reducing overconfidence , 1987 .

[19]  Rebecca A. Henry,et al.  Accuracy and confidence in group judgment , 1989 .

[20]  Bonnie A. Nardi,et al.  Twinkling Lights and Nested Loops: Distributed Problem Solving and Spreadsheet Development , 1991, Int. J. Man Mach. Stud..

[21]  Ray Fuller.  Learning to make errors: evidence from a driving task simulation , 1990 .

[22]  Dennis F. Galletta,et al.  An empirical study of spreadsheet error-finding performance , 1993 .

[23]  S. Plous,et al.  A comparison of strategies for reducing interval overconfidence in group judgments , 1995 .

[24]  I. D. Brown,et al.  Drivers' margins of safety considered as a focus for research on error , 1990 .

[25]  John D. Gould,et al.  An experimental study of people creating spreadsheets , 1987, TOIS.

[26]  Kaoru Ono,et al.  Individual judgment and group interaction: A variable perspective approach , 1988 .

[27]  Mary Sumner,et al.  The Relationship of Application Risks to Application Controls: A Study of Microcomputer-based Spreadsheet Applications , 1994 .

[28]  Dennis F. Galletta,et al.  A model of end-user computing policy: Context, process, content and compliance , 1992, Inf. Manag..

[29]  James T. Reason,et al.  Types and tokens in road accident causation , 1990 .

[30]  Malcolm King,et al.  Spreadsheet Modelling Abuse: An Opportunity for OR? , 1993 .

[31]  Robert B. Grady,et al.  Successfully applying software metrics , 1994, Computer.

[32]  Veda C. Storey,et al.  The use of spreadsheets in organizations: Determinants and consequences , 1996, Inf. Manag..