Reducing Overconfidence in Spreadsheet Development

Despite strong evidence of widespread errors, spreadsheet developers rarely subject their spreadsheets to post-development testing to reduce errors. This may be because spreadsheet developers are overconfident in the accuracy of their spreadsheets. This conjecture is plausible because overconfidence is present in a wide variety of human cognitive domains, even among experts. This paper describes two experiments in overconfidence in spreadsheet development. The first is a pilot study to determine the existence of overconfidence. The second tests a manipulation to reduce overconfidence and errors. The manipulation is modestly successful, indicating that overconfidence reduction is a promising avenue to pursue.

[1]  Carl Martin Allwood,et al.  Error Detection Processes in Statistical Problem Solving , 1984, Cogn. Sci..

[2]  S. Plous.  The psychology of judgment and decision making , 1994 .

[3]  I. D. Brown,et al.  Drivers' margins of safety considered as a focus for research on error , 1990 .

[4]  Veda C. Storey,et al.  The use of spreadsheets in organizations: Determinants and consequences , 1996, Inf. Manag..

[5]  John D. Gould,et al.  An experimental study of people creating spreadsheets , 1987, TOIS.

[6]  David G. Hendry,et al.  Creating, comprehending and explaining spreadsheets: a cognitive interpretation of what discretionary users think of the spreadsheet model , 1994, Int. J. Hum. Comput. Stud..

[7]  Kaoru Ono,et al.  Individual judgment and group interaction: A variable perspective approach , 1988 .

[8]  Mary Sumner,et al.  The Relationship of Application Risks to Application Controls: A Study of Microcomputer-based Spreadsheet Applications , 1994 .

[9]  Raymond R. Panko,et al.  Are Two Heads Better than One? (At Reducing Errors in Spreadsheet Modeling). , 1997 .

[10]  R. Kominski.  Computer Use in the United States: 1984. , 1988 .

[11]  Frank R. Clarke.  Confidence Ratings, Second‐Choice Responses, and Confusion Matrices in Intelligibility Tests , 1960 .

[12]  Maria Jean Johnstone Hall.  A risk and control-oriented study of the practices of spreadsheet application developers , 1996, Proceedings of HICSS-29: 29th Hawaii International Conference on System Sciences.

[13]  Bernard J. Baars,et al.  A New Ideomotor Theory of Voluntary Control , 1992 .

[14]  Ralph H. Sprague,et al.  Hitting the wall: errors in developing and code inspecting a 'simple' spreadsheet model , 1998, Decis. Support Syst..

[15]  L. Ross,et al.  The overconfidence effect in social prediction. , 1990, Journal of personality and social psychology.

[16]  George M. Kasper,et al.  A Theory of Decision Support System Design for User Calibration , 1996, Inf. Syst. Res..

[17]  James Shanteau,et al.  11 – JUDGMENT AND SWINE: APPROACHES AND ISSUES IN APPLIED JUDGMENT ANALYSIS , 1977 .

[18]  H. Arkes,et al.  Two methods of reducing overconfidence , 1987 .

[19]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[20]  W A Wagenaar,et al.  Does the expert know? The reliability of predictions and confidence ratings of experts , 1986 .

[21]  B. Fischhoff,et al.  Calibration of probabilities: the state of the art to 1980 , 1982 .

[22]  Ray Fuller.  Learning to make errors: evidence from a driving task simulation , 1990 .

[23]  Briony D. Pulford,et al.  Overconfidence, base rates and outcome positivity/negativity of predicted events , 1996 .

[24]  K Rumar,et al.  The basic driver error: late detection. , 1990, Ergonomics.

[25]  Guy G. Gable,et al.  Spreadsheet investment, criticality, and control , 1991, Proceedings of the Twenty-Fourth Annual Hawaii International Conference on System Sciences.

[26]  S. Plous,et al.  A comparison of strategies for reducing interval overconfidence in group judgments , 1995 .

[27]  Eric J. Johnson,et al.  Expertise and decision under uncertainty: Performance and process. , 1988 .

[28]  Gideon Keren,et al.  Improving Decisions and Judgments The Desirable versus the Feasible , 1992 .

[29]  Herbert Solomon,et al.  Two models of group behavior in the solution of eureka-type problems , 1955 .

[30]  J. Shanteau.  The Psychology of Experts An Alternative View , 1992 .

[31]  Rebecca A. Henry,et al.  Accuracy and confidence in group judgment , 1989 .

[32]  Colin Camerer,et al.  The process-performance paradox in expert judgment - How can experts know so much and predict so badly? , 1991 .

[33]  James T. Reason,et al.  Types and tokens in road accident causation , 1990 .

[34]  Malcolm King,et al.  Spreadsheet Modelling Abuse: An Opportunity for OR? , 1993 .

[35]  J S Habberley,et al.  A BEHAVIORAL STUDY OF THE COLLISION AVOIDANCE TASK IN BRIDGE WATCHKEEPING , 1986 .

[36]  Bonnie A. Nardi,et al.  Twinkling Lights and Nested Loops: Distributed Problem Solving and Spreadsheet Development , 1991, Int. J. Man Mach. Stud..