Validation of decision-aiding spreadsheets: the influence of contingency factors

This paper describes an investigation into the influence of contingency factors on the validation of spreadsheet-based decision support systems (DSS), and develops a methodological framework for validation that takes account of the effect of contingency factors. The research extends and confirms previous research by one of the authors who identified relevant contingency factors. The perceived influence of these contingency factors on validation effort was investigated by an empirical study, and the results of this study were then used to derive contingency-related guidance for validation. This was evaluated by a sample of practitioners and academics with experience in the development of spreadsheet-based DSS, and was then incorporated into a methodological framework for validation. The framework is designed to enable managers, decision-makers, project leaders and other non-OR specialists to identify the extent of validation effort likely to be required to ensure that the spreadsheet models that they build are valid. The research has implications for the validation of other types of OR models.

[1]  Hugh J. Miser,et al.  A foundational concept of science appropriate for validation in operational research , 1993 .

[2]  Peter S. Sell,et al.  Expert Systems-A Practical Introduction , 1987, IEEE Expert.

[3]  Bob Berry,et al.  Accountants and financial modelling , 1991 .

[4]  John S. Edwards,et al.  The role of OR specialists in 'do it yourself' spreadsheet development , 2000, Eur. J. Oper. Res..

[5]  Linda Leon,et al.  A Literature Survey of Spreadsheet Based MS/OR Applications: 1985–1999 , 2000 .

[6]  John G. Lynch,et al.  Validity and the research process , 1986 .

[7]  Jean-Louis Malouin,et al.  Model validation in operations research , 1983 .

[8]  S. Kanungo,et al.  Evaluation of a decision support system for credit management decisions , 2001, Decis. Support Syst..

[9]  Richard Bentley,et al.  Situated evaluation for cooperative systems , 1994, CSCW '94.

[10]  Paul N. Finlay,et al.  The Validation of Expert Systems — Contrasts with Traditional Methods , 1988 .

[11]  Muhittin Oral,et al.  Model legitimisation in operational research , 1996 .

[12]  Paul N. Finlay,et al.  Validity of Decision Support Systems: Towards a Validation Methodology , 1997 .

[13]  J. D. Ryan.  Handbook of Systems Analysis — Craft Issues and Procedural Choices , 1988 .

[14]  Paul N. Finlay,et al.  A survey of contingency factors affecting the validation of end-user spreadsheet-based decision support systems , 2000, J. Oper. Res. Soc..

[15]  Maurice Landry,et al.  Revisiting the issue of model validation in OR: An epistemological view , 1993 .

[16]  John D. Gould,et al.  An experimental study of people creating spreadsheets , 1987, TOIS.

[17]  Paul N. Finlay,et al.  The Paucity of Model Validation in Operational Research Projects , 1987 .

[18]  John Davies,et al.  Knowledge management and the framing of information: a contribution to OR/MS practice and pedagogy , 2001, J. Oper. Res. Soc..

[19]  Timothy Paul Cronan,et al.  Applications development by end-users: can quality be improved? , 2000, Decis. Support Syst..

[20]  Ralph H. Sprague,et al.  Hitting the wall: errors in developing and code inspecting a 'simple' spreadsheet model , 1998, Decis. Support Syst..

[21]  Michel Klein,et al.  Expert systems - a decision support approach with applications in management and finance , 1990, Insight series in artificial intelligence.

[22]  P. Strong,et al.  The case for qualitative research , 1992 .

[23]  Muhittin Oral,et al.  The facets of the modeling and validation process in operations research , 1993 .