A Comparison of Laboratory and Field Research in the Study of Electronic Meeting Systems

Abstract:Research into the use of Electronic Meeting Systems (ems) has rapidly increased over the past few years. However, ems laboratory experiments have often drawn very different conclusions about the effects of ems use than have ems field studies—conclusions that at first appear inconsistent. By examining the differences in the design of prior ems studies, we attempt to better understand the factors that affect the use of ems technology. It is our contention that these differences in findings are not inconsistent, but rather they reflect different situations that researchers have studied. We identify 24 potentially important differences in organizational contexts, group characteristics, tasks, and ems environments. In planning future research, ems researchers need to make explicit design decisions for each of these aspects, and to consider how those choices may affect research results. An analysis of these differences suggests several approaches to enhance the design of future laboratory experiments a...

[1]  Allen S. Lee A Scientific Methodology for MIS Case Studies , 1989, MIS Q..

[2]  F. van Schaik,et al.  Effectiveness of decision support systems , 1988 .

[3]  R. Gallupe,et al.  Suppressing the contribution of the group's best member: is GDSS use appropriate for all group tasks? , 1990, Twenty-Third Annual Hawaii International Conference on System Sciences.

[4]  Ramesh Sharda,et al.  Decision support system effectiveness: a review and an empirical test , 1988 .

[5]  V. Smith Microeconomic Systems as an Experimental Science , 1982 .

[6]  James D. McKeen,et al.  Enhancing Computer-Mediated Communication: An experimental investigation into the use of a Group Decision Support System for face-to-face versus remote meetings , 1990, Inf. Manag..

[7]  Richard T. Watson,et al.  Group decision support systems: the cultural factor , 1989, ICIS '89.

[8]  Tung Bui,et al.  Relation between GDSS use and group task complexity: an experimental study , 1990, Twenty-Third Annual Hawaii International Conference on System Sciences.

[9]  D R Vogel,et al.  Automated planning support. Using computers to enhance group decision making. , 1989, Administrative radiology : AR.

[10]  H. Rittel,et al.  Dilemmas in a general theory of planning , 1973 .

[11]  Gerardine DeSanctis,et al.  ICIS Paper: Methodological Issues in Experimental IS Research: Experiences and Recommendations , 1985, MIS Q..

[12]  Sirkka L. Jarvenpaa,et al.  Computer Support for Meetings of Groups Working on Unstructured Problems: A Field Experiment , 1988, MIS Q..

[13]  Gerardine DeSanctis,et al.  A foundation for the study of group decision support systems , 1987 .

[14]  J. George,et al.  Ad hoc versus established groups in an electronic meeting system environment , 1990, Twenty-Third Annual Hawaii International Conference on System Sciences.

[15]  Frans D. VanSchaik,et al.  Effectiveness of decision support systems , 1990, Twenty-Third Annual Hawaii International Conference on System Sciences.

[16]  William Remus,et al.  Graduate students as surrogates for managers in experiments on business decision making , 1986 .

[17]  Jay F. Nunamaker,et al.  A Study of Collaborative Group Work With and Without Computer-Based Support , 1990, Inf. Syst. Res..

[18]  Jay F. Nunamaker,et al.  An experimental investigation of the effects of group size in an electronic meeting environment , 1990, IEEE Trans. Syst. Man Cybern..

[19]  Leonard Michael Jessup,et al.  Decision Making in an Automated Environment: The Effects of Anonymity and Proximity with a Group Decision Support System* , 1991 .

[20]  M. E. Shaw Group dynamics : the psychology of small group behavior , 1971 .

[21]  William Remus,et al.  Using students as subjects in experiments on decision support systems , 1989, [1989] Proceedings of the Twenty-Second Annual Hawaii International Conference on System Sciences. Volume III: Decision Support and Knowledge Based Systems Track.

[22]  Robert P. Bostrom,et al.  An empirical investigation of the impact of computer support on group development , 1990, Twenty-Third Annual Hawaii International Conference on System Sciences.

[23]  Anne T. McCartt,et al.  Evaluating group decision support system effectiveness: A performance study of decision conferencing , 1989, Decis. Support Syst..

[24]  Frank F. Land,et al.  Viewpoint: choosing appropriate information systems research methodologies , 1987, CACM.

[25]  Gerardine DeSanctis,et al.  Computer-Based Support for Group Problem-Finding: An Experimental Investigation , 1988, MIS Q..

[26]  Kwok Kee Wei,et al.  Does GDSS promote more democratic decision-making?-The Singapore experiment , 1990, Twenty-Third Annual Hawaii International Conference on System Sciences.

[27]  V. Sambamurthy,et al.  An experimental evaluation of GDSS effects on group performance during stakeholder analysis , 1990, Twenty-Third Annual Hawaii International Conference on System Sciences.

[28]  M. Diehl,et al.  Productivity loss in brainstorming groups: Toward the solution of a riddle. , 1987 .

[29]  Jay F. Nunamaker,et al.  Electronic Meeting System Experience at IBM , 1989, J. Manag. Inf. Syst..

[30]  Ilze Zigurs,et al.  A Study of Influence in Computer-Mediated Group Decision Making , 1988, MIS Q..

[31]  Gerardine DeSanctis,et al.  Comprehensiveness and restrictiveness in group decision heuristics: effects of computer support on consensus decision making , 1989, ICIS '89.

[32]  Kenneth L. Kraemer,et al.  Computer-based systems for cooperative work and group decision making , 1988, CSUR.

[33]  Robert D. McPhee,et al.  Group decision‐making as a structurational process , 1985 .

[34]  G. Kreweras Creating the corporate future: Russell L. ACKOFF Wiley, New York, 1981, xi + 297 pages, £10.85 , 1982 .

[35]  G. Huber Cognitive Style as a Basis for MIS and DSS Designs: Much ADO About Nothing? , 1983 .

[36]  J. McGrath Groups: Interaction and Performance , 1984 .

[37]  J. Valacich,et al.  Effects of anonymity and evaluative tone on idea generation in computer-mediated groups , 1990 .

[38]  Jay Hall,et al.  A comparison of decision-making performances in established and ad hoc groups. , 1966 .

[39]  Izak Benbasat,et al.  The Case Research Strategy in Studies of Information Systems , 1987, MIS Q..

[40]  Jay F. Nunamaker,et al.  Facilitating Group Creativity: Experience with a Group Decision Support System , 1987, J. Manag. Inf. Syst..

[41]  Randall Steeb,et al.  A Computer-Based Interactive System for Group Decisionmaking , 1981, IEEE Transactions on Systems, Man, and Cybernetics.

[42]  Jay F. Nunamaker,et al.  Computer-Aided Deliberation: Model Management and Group Decision Support: Special Focus Article , 1988, Oper. Res..

[43]  J. Hackman,et al.  Interventions into group process: An approach to improving the effectiveness of groups , 1974 .

[44]  Kenneth L. Kraemer,et al.  The impact of technological support on groups: An assessment of the empirical research , 1989, Decis. Support Syst..

[45]  Jay F. Nunamaker,et al.  Information Technology to Support Electronic Meetings , 1988, MIS Q..

[46]  Jay F. Nunamaker,et al.  Stakeholder identification and assumption surfacing in small groups: an experimental study , 1989, [1989] Proceedings of the Twenty-Second Annual Hawaii International Conference on System Sciences. Volume III: Decision Support and Knowledge Based Systems Track.

[47]  R. Mason Challenging strategic planning assumptions , 1981 .

[48]  Jay F. Nunamaker,et al.  Using Two Different Electronic Meeting System Tools for the Same Task: An Experimental Comparison , 1990, J. Manag. Inf. Syst..

[49]  M. Gordon,et al.  The “Science of the Sophomore” Revisited: from Conjecture to Empiricism , 1986 .

[50]  Robert P. Bostrom,et al.  Identifying the root causes of data and systems planning problems: an application of the PLEXSYS electronic meeting support system , 1990, Twenty-Third Annual Hawaii International Conference on System Sciences.

[51]  Jay F. Nunamaker,et al.  Experience at IBM with group support systems: A field study , 1989, Decis. Support Syst..

[52]  Leonard M. Jessup,et al.  The Effects of Anonymity on GDSS Group Process with an Idea-Generating Task , 1990, MIS Q..

[53]  Jay F. Nunamaker,et al.  Bringing automated support to large groups: The Burr-Brown experience , 1990, Inf. Manag..

[54]  Sirkka L. Jarvenpaa The importance of laboratory experimentation in IS research (technical correspondence) , 1988, CACM.

[55]  B. E. Wynne,et al.  The effect of group size and computer support on group idea generation for creativity tasks: an experimental evaluation using a repeated measures design , 1989 .

[56]  Craig K. Tyran,et al.  An Evaluation of Electronic Meeting Systems to Support Strategic Management , 1990, ICIS.

[57]  George P. Huber,et al.  The nature of organizational decision making and the desig n of decision support systems , 1981 .

[58]  S. Kiesler,et al.  Group processes in computer-mediated communication☆ , 1986 .

[59]  Gerardine DeSanctis,et al.  Using a GDSS to Facilitate Group Consensus: Some Intended and Unintended Consequences , 1988, MIS Q..

[60]  Leonard Adelman,et al.  Real-Time Computer Support for Decision Analysis in a Group Setting: Another Class of Decision Support Systems , 1984 .

[61]  Bonnie Kaplan,et al.  Combining Qualitative and Quantitative Methods in Information Systems Research: A Case Study , 1988, MIS Q..

[62]  Jay F. Nunamaker,et al.  Electronic Meeting Support: The GroupSystems Concept , 1991, Int. J. Man Mach. Stud..