Evaluating meeting support tools

Many attempts are underway for developing meeting support tools, but less attention is paid to the evaluation of meetingware. This article describes the development and testing of an instrument for evaluating meeting tools. First, we specified the object of evaluation—meetings—by means of a set of input, process, and outcome factors. Then, we designed the process of evaluation, consisting of, first, the generation of meeting behavior in the form of a controlled series of meetings within the context of a project and, second, the measurement of the identified meeting factors. To measure these factors, a rating scale, questionnaires, and information flow analysis were used. Next, the instrument was tested, and the factors for successful meetings were determined in 13 projects in which four participants had to meet four times. The evaluation instrument proved to be a reliable and useful aid for the development and improvement of meeting tools.

[1]  John Short,et al.  The social psychology of telecommunications , 1976 .

[2]  Mary Beth Rosson,et al.  Evaluating computer-supported cooperative work: models and frameworks , 2004, CSCW.

[3]  Nigel Shadbolt,et al.  Knowledge Engineering and Management , 2000 .

[4]  Bongsik Shin,et al.  The Dynamic Effects of Group Support Systems on Group Meetings , 2002, J. Manag. Inf. Syst..

[5]  Wilfried Post,et al.  A research environment for meeting behavior , 2004 .

[6]  E. A. Fleishman,et al.  Team Dimensions: Their Identity, Their Measurement and Their Relationships , 1985 .

[7]  Pedro Antunes,et al.  Perceived Value: A Low-Cost Approach to Evaluate Meetingware , 2003, CRIWG.

[8]  John Karat,et al.  User-Centered Software Evaluation Methodologies , 1997 .

[9]  Eduardo Salas,et al.  Team Effectiveness and Decision Making in Organizations , 1995 .

[10]  Starr Roxanne Hiltz,et al.  An Assessment of Group Support Systems Experimental Research: Methodology and Results , 1998, J. Manag. Inf. Syst..

[11]  Guus Schreiber,et al.  Knowledge Engineering and Management: The CommonKADS Methodology , 1999 .

[12]  I. Steiner.  Group process and productivity , 1972 .

[13]  Michael A. West,et al.  Team Effectiveness in Organizations , 1998 .

[14]  Stephanie C Payne,et al.  Measuring team-related expertise in complex environments. , 1998 .

[15]  J. McGrath,et al.  Groups Interacting with Technology: Ideas, Evidence, Issues and an Agenda , 1993 .

[16]  J. Levine,et al.  Progress in Small Group Research , 1990 .

[17]  Barbara M. Moskal,et al.  Engineering Design: Examining the Impact of Gender and the Team's Gender Composition , 2003 .

[18]  M.A.A. Huis in 't Veld,et al.  E-Magine: An Evaluation Method to Assess Groups using ICT , 2000 .

[19]  G. Fenwick,et al.  Effect of Gender Composition on Group Performance , 2001 .

[20]  Jay F. Nunamaker,et al.  Lessons from a Dozen Years of Group Support Systems Research: A Discussion of Lab and Field Findings , 1996, J. Manag. Inf. Syst..

[21]  Guus Schreiber,et al.  A Case Study in Using Protégé-2000 as a Tool for CommonKADS , 2000, EKAW.

[22]  F. Dattilio,et al.  Groups: Theory and Experience , 1986 .

[23]  Eduardo Salas,et al.  Making decisions under stress: Implications for individual and team training. , 1998 .

[24]  Anthony W. K. Gaillard.  Stress, productiviteit en gezondheid , 2003 .

[25]  M. E. Shaw.  Group dynamics : the psychology of small group behavior , 1971 .

[26]  Dick de Waard,et al.  The measurement of drivers' mental workload , 1996 .

[27]  J. Valacich,et al.  Group Support Systems: New Perspectives , 1992 .

[28]  H. P. Sims,et al.  Team Work and Group Dynamics , 1998 .

[29]  Karl E. Weick,et al.  Managing the unexpected: Assuring high performance in an age of complexity. , 2001 .