“Horses for courses”: A stakeholder approach to the evaluation of GDSSs

Evaluation of the performance of GDSSs has been dominated by an experimental and laboratory based approach. Other writers have argued for evaluation to be based in the “real-world” of decision making teams. The evaluation criteria have tended to ignore many of the issues that would be paramount for some of the stakeholders in the evaluation process. This article seeks to explore the criteria that might be used by a wide variety of stakeholders, including developers, facilitators, clients, key actors, vendors, as well as academics. By drawing together the criteria associated with all of the stakeholders we discover a broader, and possibly more thorough, framework for evaluation. The evaluation of any particular GDSS in relation to other GDSSs can then be seen in the context of contingent weighting applied to each of the criteria where each GDSS is able to be seen in its best light and in relation to its declared aims.This article argues for a more eclectic and contingent approach to the evaluation of GDSSs which will encourage their future development to be clearer about purpose and the boundaries of their use.

[1]  Colin Eden,et al.  On evaluating the performance of 'wide-band' GDSS's , 1995 .

[2]  Jay F. Nunamaker,et al.  A Comparison of Laboratory and Field Research in the Study of Electronic Meeting Systems , 1990, J. Manag. Inf. Syst..

[3]  Jay F. Nunamaker,et al.  Information Technology to Support Electronic Meetings , 1988, MIS Q..

[4]  E. Suchman.  Evaluative Research: Principles and Practice in Public Service and Social Action Progr , 1968 .

[5]  George P. Huber,et al.  Issues in the Design of Group Decision Support Systems , 1984, MIS Q..

[6]  Colin Eden,et al.  Exploring and negotiating collective action through computer-aided cognitive mapping , 1992 .

[7]  R. Dyson,et al.  Reducing Weight Flexibility in Data Envelopment Analysis , 1988 .

[8]  Robert Johansen.  TEAMS FOR TOMORROW , 1991 .

[9]  Pedro Sánchez,et al.  Collaborative technology and group process feedback: their impact on interactive sequences in meetings , 1990, CSCW '90.

[10]  Kenneth L. Kraemer,et al.  The impact of technological support on groups: An assessment of the empirical research , 1989, Decis. Support Syst..

[11]  R. Bales.  Interaction process analysis , 1976 .

[12]  P. Clark.  Action research and organizational change , 1972 .

[13]  Colin Eden,et al.  Evaluating Strategy — Its Role Within the Context of Strategic Control , 1993 .

[14]  Izak Benbasat,et al.  The Information Systems Research Challenge: Experimental Research Methods , 1990 .

[15]  Jay F. Nunamaker,et al.  Bringing automated support to large groups: The Burr-Brown experience , 1990, Inf. Manag..

[16]  Jay F. Nunamaker,et al.  A Study of Collaborative Group Work With and Without Computer-Based Support , 1990, Inf. Syst. Res..

[17]  L. Phillips.  People-centred group decision support , 1989 .

[18]  Jay F. Nunamaker,et al.  Computer-Aided Deliberation: Model Management and Group Decision Support: Special Focus Article , 1988, Oper. Res..

[19]  R. Yin.  Case Study Research: Design and Methods , 1984 .

[20]  Peter M. Bennett,et al.  Decision-aiding demonstrated! , 1989 .

[21]  S. Fiske,et al.  The Handbook of Social Psychology , 1935 .

[22]  Paul N. Finlay,et al.  The Paucity of Model Validation in Operational Research Projects , 1987 .

[23]  R. Johansen.  Teams for tomorrow (groupware) , 1991, Proceedings of the Twenty-Fourth Annual Hawaii International Conference on System Sciences.

[24]  L. Floyd Lewis,et al.  Facilitator : a microcomputer decision support system for small groups , 1982 .

[25]  Jay F. Nunamaker,et al.  An experimental investigation of the effects of group size in an electronic meeting environment , 1990, IEEE Trans. Syst. Man Cybern..

[26]  Sirkka L. Jarvenpaa,et al.  Computer Support for Meetings of Groups Working on Unstructured Problems: A Field Experiment , 1988, MIS Q..

[27]  J. Valacich,et al.  Group Support Systems: New Perspectives , 1992 .

[28]  Allen S. Lee.  A Scientific Methodology for MIS Case Studies , 1989, MIS Q..

[29]  Fran Ackermann.  Strategic direction through burning issues , 1992 .

[30]  Fran Ackermann,et al.  Strategy development and implementation - the role of a group decision support system , 1992 .

[31]  Jean-Louis Malouin,et al.  Model validation in operations research , 1983 .

[32]  Randall Steeb,et al.  A Computer-Based Interactive System for Group Decisionmaking , 1981, IEEE Transactions on Systems, Man, and Cybernetics.

[33]  G. Susman,et al.  An Assessment of the Scientific Merits of Action Research. , 1978 .

[34]  R. Bostrom,et al.  Group Facilitation and Group Support Systems , 1993 .

[35]  M. S. Ackerman,et al.  Answer Garden: a tool for growing organizational memory , 2015, COCS '90.

[36]  Gerardine DeSanctis,et al.  Using computing to improve the quality team process: some initial observations from the IRS-Minnesota project , 1991, Proceedings of the Twenty-Fourth Annual Hawaii International Conference on System Sciences.

[37]  Saul I. Gass,et al.  OR Practice - The Army Manpower Long-Range Planning System , 1988, Oper. Res..

[38]  Starr Roxanne Hiltz,et al.  Computer Support for Group Versus Individual Decisions , 1982, IEEE Trans. Commun..

[39]  Abraham Charnes,et al.  Measuring the efficiency of decision making units , 1978 .

[40]  Valerie Belton,et al.  Demystifying DEA — A Visual Interactive Approach Based on Multiple Criteria Analysis , 1993 .

[41]  Gerardine DeSanctis,et al.  Using a GDSS to Facilitate Group Consensus: Some Intended and Unintended Consequences , 1988, MIS Q..

[42]  Fran Ackermann,et al.  Issues in computer and non-computer supported GDSSs , 1994, Decis. Support Syst..

[43]  Frank Land,et al.  Knowledge based management support systems , 1989 .

[44]  Paul Gray,et al.  Assessing GDSS empirical research , 1990 .

[45]  Colin Eden,et al.  Tackling Strategic Problems: The Role of Group Decision Support , 1991 .

[46]  Stewart Clegg,et al.  Handbook of organization studies , 1997 .

[47]  Gerardine DeSanctis,et al.  Computer-Based Support for Group Problem-Finding: An Experimental Investigation , 1988, MIS Q..

[48]  Jay F. Nunamaker,et al.  Using Two Different Electronic Meeting System Tools for the Same Task: An Experimental Comparison , 1990, J. Manag. Inf. Syst..

[49]  E. Rogers,et al.  Research methods and the new media , 1988 .

[50]  Martin Read,et al.  On-line group process support , 1993 .

[51]  F. Ackermann.  Participants' perceptions on the role of facilitators using Group Decision Support Systems , 1996 .

[52]  C. Eden.  A framework for thinking about Group Decision Support Systems (GDSS) , 1992 .

[53]  Colin Eden,et al.  Appendix: A client's perspective , 1990 .

[54]  Colin Eden,et al.  Action research for the study of organizations , 1996 .