What we know about spreadsheet errors

Although spreadsheet programs are used for small "scratchpad" applications, they are also used to develop many large applications. In recent years, we have learned a good deal about the errors that people make when they develop spreadsheets. In general, errors seem to occur in a few percent of all cells, meaning that for large spreadsheets, the issue is how many errors there are, not whether an error exists. These error rates, although troubling, are in line with those in programming and other human cognitive domains. In programming, we have learned to follow strict development disciplines to eliminate most errors. Surveys of spreadsheet developers indicate that spreadsheet creation, in contrast, is informal, and few organizations have comprehensive policies for spreadsheet development. Although prescriptive articles have focused on such

[1]  Joline Morrison,et al.  Factors influencing risks and outcomes in end-user development , 1996, Proceedings of HICSS-29: 29th Hawaii International Conference on System Sciences.

[2]  J S Habberley,et al.  A BEHAVIORAL STUDY OF THE COLLISION AVOIDANCE TASK IN BRIDGE WATCHKEEPING , 1986 .

[3]  Gordon B. Davis,et al.  Caution: user developed systems can be dangerous to your organization , 1989 .

[4]  Bonnie A. Nardi,et al.  Twinkling Lights and Nested Loops: Distributed Problem Solving and Spreadsheet Development , 1991, Int. J. Man Mach. Stud..

[5]  I. D. Brown,et al.  Drivers' margins of safety considered as a focus for research on error , 1990 .

[6]  John D. Gould,et al.  An experimental study of people creating spreadsheets , 1987, TOIS.

[7]  Joline Morrison,et al.  Using a structured design approach to reduce risks in end user spreadsheet development , 2000, Inf. Manag..

[8]  Glenford J. Myers,et al.  A controlled experiment in program testing and code walkthroughs/inspections , 1978, CACM.

[9]  T. Gilovich,et al.  How We Know What Isn't So: The Fallibility of Human Reason in Everyday Life , 1991 .

[10]  David Freeman How to Make Spreadsheets Error-Proof , 1996 .

[11]  R.C. Linger,et al.  Cleanroom process model , 1994, IEEE Software.

[12]  George M. Kasper,et al.  A Theory of Decision Support System Design for User Calibration , 1996, Inf. Syst. Res..

[13]  Michael E. Fagan Design and Code Inspections to Reduce Errors in Program Development , 1976, IBM Syst. J..

[14]  Carol V. Brown,et al.  Perceived risks and management actions: differences in end-user application development across functional groups , 1996, Proceedings of HICSS-29: 29th Hawaii International Conference on System Sciences.

[15]  Eric J. Johnson,et al.  Expertise and decision under uncertainty: Performance and process. , 1988 .

[16]  Raymond R. Panko,et al.  Are Two Heads Better than One? (At Reducing Errors in Spreadsheet Modeling). , 1997 .

[17]  Dennis F. Galletta,et al.  A model of end-user computing policy: Context, process, content and compliance , 1992, Inf. Manag..

[18]  Marilyn M. Mantei,et al.  Computerized financial planning: discovering cognitive difficulties in model building , 1988 .

[19]  Barry D. Floyd,et al.  Errors in Spreadsheet Use , 1987 .

[20]  Linda Flower,et al.  The Dynamics of Composing : Making Plans and Juggling Constraints , 1980 .

[21]  E. Thorndike “Animal Intelligence” , 1898, Nature.

[22]  Jerry Waxman,et al.  A Study of Three Database Query Languages , 1978, JCDKB.

[23]  Malcolm King,et al.  Spreadsheet Modelling Abuse: An Opportunity for OR? , 1993 .

[24]  Sebastiano Bagnara,et al.  Human Error Detection Processes , 1987, Int. J. Man Mach. Stud..

[25]  Frank R. Clarke Confidence Ratings, Second‐Choice Responses, and Confusion Matrices in Intelligibility Tests , 1960 .

[26]  Raymond R. Panko,et al.  Two corpuses of spreadsheet errors , 2000, Proceedings of the 33rd Annual Hawaii International Conference on System Sciences.

[27]  Henry C. Lucas,et al.  Spreadsheet analysis and design , 1989, CACM.

[28]  Dennis F. Galletta,et al.  Spreadsheet Presentation and Error Detection: An Experimental Study , 1997, J. Manag. Inf. Syst..

[29]  Philip Johnson,et al.  Exploring the effectiveness of formal technical review factors with csrs, a collaborative software review system , 1996 .

[30]  Hayes identifying the organization of wi iiing processes , 1980 .

[31]  R. Mayer Thinking, Problem Solving, Cognition , 1983 .

[32]  David Woods,et al.  SOME RESULTS ON OPERATOR PERFORMANCE IN EMERGENCY EVENTS , 1984 .

[33]  Brian Knight,et al.  Quality control in spreadsheets: a software engineering-based approach to spreadsheet development , 2000, Proceedings of the 33rd Annual Hawaii International Conference on System Sciences.

[34]  Arthur E. Ferdinand Systems, Software, and Quality Engineering: Applying Defect Behavior Theory to Programming , 1993 .

[35]  C. I. Howarth,et al.  The relationship between objective risk, subjective risk and behaviour , 1988 .

[36]  Victor R. Basili,et al.  Four Applications of a Software Data Collection and Analysis Methodology , 1986 .

[37]  Herbert Solomon,et al.  Two models of group behavior in the solution of eureka-type problems , 1955 .

[38]  B. Fischhoff,et al.  Training for calibration. , 1980 .

[39]  Philip Banyard,et al.  Thinking & problem solving , 1991 .

[40]  Boris Beizer,et al.  Software Testing Techniques , 1983 .

[41]  B. Fischhoff,et al.  Calibration of probabilities: the state of the art to 1980 , 1982 .

[42]  A. Edmondson Learning from Mistakes is Easier Said Than Done: Group and Organizational Influences on the Detection and Correction of Human Error , 1996 .

[43]  Raymond R. Panko,et al.  Applying Code Inspection to Spreadsheet Testing , 1999, J. Manag. Inf. Syst..

[44]  Raymond R. Panko,et al.  Spreadsheets on trial: a survey of research on spreadsheet risks , 1996, Proceedings of HICSS-29: 29th Hawaii International Conference on System Sciences.

[45]  Mary Sumner,et al.  The Relationship of Application Risks to Application Controls: A Study of Microcomputer-based Spreadsheet Applications , 1994 .

[46]  Dennis F. Galletta,et al.  An empirical study of spreadsheet error-finding performance , 1993 .

[47]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[48]  W A Wagenaar,et al.  Does the expert know? The reliability of predictions and confidence ratings of experts , 1986 .

[49]  Bonnie A. Nardi,et al.  A Small Matter of Programming: Perspectives on End User Computing , 1993 .

[50]  Patrick Rabbitt,et al.  An elementary preliminary taxonomy for some errors in laboratory choice RT tasks , 1970 .

[51]  Margaret Tan,et al.  Quantitative and qualitative errors in spreadsheet development , 1997, Proceedings of the Thirtieth Hawaii International Conference on System Sciences.

[52]  Henry C. Lucas,et al.  Toward a logical/physical theory of spreadsheet modeling , 1992, TOIS.

[53]  Harry C. Benham,et al.  Structured Techniques for Successful End User Spreadsheets , 1993 .

[54]  G. Miller,et al.  Cognitive science. , 1981, Science.

[55]  Raymond J. Butler,et al.  Is this spreadsheet a tax evader? How HM Customs and Excise test spreadsheet applications , 2000, Proceedings of the 33rd Annual Hawaii International Conference on System Sciences.

[56]  David G. Hendry,et al.  Creating, comprehending and explaining spreadsheets: a cognitive interpretation of what discretionary users think of the spreadsheet model , 1994, Int. J. Hum. Comput. Stud..

[57]  Baruch Fischhoff,et al.  Calibration of Probabilities: The State of the Art , 1977 .

[58]  James Reason,et al.  Human Error , 1990 .

[59]  Ray Fuller Learning to make errors: evidence from a driving task simulation , 1990 .

[60]  Raymond R. Panko End user computing: management applications and technology , 1988 .

[61]  Ware Myers,et al.  Measures for Excellence: Reliable Software on Time, Within Budget , 1991 .

[62]  Bernard J. Baars,et al.  A New Ideomotor Theory of Voluntary Control , 1992 .

[63]  Ralph H. Sprague,et al.  Hitting the wall: errors in developing and code inspecting a 'simple' spreadsheet model , 1998, Decis. Support Syst..

[64]  James Shanteau,et al.  11 – JUDGMENT AND SWINE: APPROACHES AND ISSUES IN APPLIED JUDGMENT ANALYSIS , 1977 .

[65]  H. Arkes,et al.  Two methods of reducing overconfidence , 1987 .

[66]  David Dunning,et al.  The overconfidence effect in social prediction. , 1990 .

[67]  Carl Martin Allwood,et al.  Error Detection Processes in Statistical Problem Solving , 1984, Cogn. Sci..

[68]  S. Plous The psychology of judgment and decision making , 1994 .

[69]  S. Ditlea,et al.  Spreadsheets can be hazardous to your health , 1987 .

[70]  Maria Jean Johnstone Hall A risk and control-oriented study of the practices of spreadsheet application developers , 1996, Proceedings of HICSS-29: 29th Hawaii International Conference on System Sciences.