An empirical study of spreadsheet error-finding performance

Several well-founded concerns exist about the integrity and validity of electronic spreadsheets. Thirty CPAs and 30 MBA students volunteered to seek up to two errors planted in each of six spread-sheets to discover if expertise in the domain of accounting and the spreadsheet program would facilitate error-finding performance. Subjects only found, on average, about 55% of the simple, conspicuous errors on the small spreadsheets. As expected, both accounting and spreadsheeting expertise contributed to the subjects' error-finding rate, and those who performed this task with both types of expertise found the largest number of errors in the shortest time. Interestingly, while CPAs were more accurate than non-CPAs in finding accounting-related errors, spreadsheet experts did not outperform novices in finding spreadsheet formula errors. Also, while spreadsheet expertise contributed to greater speed, accounting expertise did not. Future research would further investigate the contribution of spreadsheet expertise to the error-finding task. Practitioners should be aware of the difficulties in finding even simple errors, and develop training programs to facilitate spreadsheet auditors' performance.

[1]  S. Ditlea,et al.  Spreadsheets can be hazardous to your health , 1987 .

[2]  Jane M. Mackay,et al.  Training needs of novices and experts with referent experience and task domain knowledge , 1991, Inf. Manag..

[3]  Gary W. Dickson,et al.  Problem solving for effective systems analysis: an experimental exploration , 1983, CACM.

[4]  Frederick P. Brooks,et al.  No Silver Bullet: Essence and Accidents of Software Engineering , 1987 .

[5]  John A. Gould,et al.  Some Psychological Evidence on How People Debug Computer Programs , 1975, Int. J. Man Mach. Stud..

[6]  Gary Moore,et al.  Expansion and Control of End-User Computing , 1988, J. Manag. Inf. Syst..

[7]  Wiliam F. Messier.  THE EFFECT OF EXPERIENCE AND FIRM TYPE ON MATERIALITY DISCLOSURE JUDGMENTS , 1983 .

[8]  Kate Ehrlich,et al.  An empirical investigation of the tacit plan knowledge in programming , 1984 .

[9]  Michelene T. H. Chi,et al.  Expertise in Problem Solving. , 1981 .

[10]  Gary M. Olson,et al.  Comprehension differences in debugging by skilled and novice programmers , 1986 .

[11]  Dorothea P. Simon,et al.  Individual Differences in Solving Physics Problems (1978) , 1978 .

[12]  Dennis F. Galletta,et al.  A Role Theory Perspective on End-User Development , 1990, Inf. Syst. Res..

[13]  Nicholas P. Vitalari,et al.  From the Research Director , 1986, Data Base.

[14]  Joyce J. Elam,et al.  A Comparative Study of How Experts and Novices Use a Decision Aid to Solve Problems in Complex Knowledge Domains , 1992, Inf. Syst. Res..

[15]  Dennis F. Galletta,et al.  A model of end-user computing policy: Context, process, content and compliance , 1992, Inf. Manag..

[16]  Stephen E. Fienberg,et al.  Discrete Multivariate Analysis: Theory and Practice , 1976 .

[17]  Barry D. Floyd,et al.  Errors in Spreadsheet Use , 1987 .

[18]  Eric J. Johnson,et al.  Expertise and decision under uncertainty: Performance and process. , 1988 .

[19]  Ira Solomon,et al.  Experience, expertise and expert-performance research in public accounting , 1989 .

[20]  Barry L. Lewis,et al.  Determinants of Auditor Expertise , 1990 .

[21]  Maryam Alavi,et al.  Strategies for End-User Computing: An Integrative Framework , 1987, J. Manag. Inf. Syst..

[22]  Ronald Pearson.  Lies, damned lies, and spreadsheets , 1988 .

[23]  Kate Ehrlich,et al.  Knowledge and processes in the comprehension of computer programs. , 1988 .

[24]  H. Simon,et al.  Perception in chess , 1973 .

[25]  Jane M. Mackay,et al.  An Empirical Investigation of the Effects of Decision Aids on Problem-Solving Processes* , 1992 .

[26]  Gordon B. Davis,et al.  Caution: user developed systems can be dangerous to your organization , 1989 .

[27]  John F. Rockart,et al.  The management of end user computing , 1983, CACM.

[28]  John D. Gould,et al.  Reading from CRT Displays Can Be as Fast as Reading from Paper , 1987 .

[29]  John D. Gould,et al.  An experimental study of people creating spreadsheets , 1987, TOIS.

[30]  F GallettaDennis,et al.  A model of end-user computing policy , 1992 .

[31]  Edward A. Youngs.  Human Errors in Programming , 1974, Int. J. Man Mach. Stud..

[32]  Stanley Letovsky,et al.  Cognitive processes in program comprehension , 1986, J. Syst. Softw..

[33]  Douglas R. Vogel,et al.  An investigation of the information center from the user's perspective , 1985, DATB.

[34]  Curtis R. Cook,et al.  Effects of programming experience in debugging semantic errors , 1989, J. Syst. Softw..

[35]  Donald P. Ballou,et al.  Implications of data quality for spreadsheet analysis , 1987, DATB.

[36]  Robert Libby,et al.  Expertise And Auditors Judgments Of Conjunctive Events , 1986 .

[37]  V Barnes,et al.  Reading Is Slower from CRT Displays than from Paper: Attempts to Isolate a Single-Variable Explanation , 1987, Human factors.

[38]  B. Adelson,et al.  The Role of Domain Expenence in Software Design , 1985, IEEE Transactions on Software Engineering.

[39]  Iris Vessey,et al.  Expertise in Debugging Computer Programs: A Process Analysis , 1984, Int. J. Man Mach. Stud..

[40]  Herbert A. Simon,et al.  Models of Competence in Solving Physics Problems , 1980, Cogn. Sci..

[41]  B. Adelson.  Problem solving and the development of abstract categories in programming languages , 1981, Memory & cognition.

[42]  Martha E. Crosby,et al.  How do we read algorithms? A case study , 1990, Computer.

[43]  Elliot Soloway,et al.  Preprogramming Knowledge: A Major Source of Misconceptions in Novice Programmers , 1985, Hum. Comput. Interact..

[44]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[45]  Kate Ehrlich,et al.  Empirical Studies of Programming Knowledge , 1984, IEEE Transactions on Software Engineering.

[46]  William F. Wright,et al.  Internal Control Judgments And Effects Of Experience - Replications And Extensions , 1982 .

[47]  Mary Sumner,et al.  Information systems strategy and end-user application development , 1987, DATB.

[48]  B. Schneiderman,et al.  Designing the User Interface. Strategies for Effective Human-Computer Interaction , 1992 .

[49]  Wladyslaw M. Turski,et al.  No Silver Bullet - Essence and Accidents of Software Engineering - Response , 1986, IFIP Congress.