Evaluating decision making performance in the GDSS environment using data envelopment analysis

We provide a framework to evaluate decision making performance with Group Decision Support Systems (GDSSs) using an overall performance indicator. The indicator is constructed using Data Envelopment Analysis (DEA), which measures the economic efficiency of a decision-making process, or how efficiently a GDSS user converts decision resources into decision outcomes. To illustrate how our framework can be applied, we conduct an experiment that manipulates three contextual factors: communication mode, incentive structure, and group leadership. Within these contexts, we obtain an individual DEA decision efficiency score during the GDSS decision process, where participants are engaged in a mixed-motive task. We find that the communication mode, incentive structure, and interaction between the two significantly influence decision efficiency within this GDSS setting. However, the leadership structure does not have a significant impact on decision efficiency. The current study presents the first application of DEA in the area of GDSS. We illustrate how DEA is useful for evaluating GDSS-supported decision efficiency. The results suggest implications that guide the design of the next generation GDSS to support individuals within collaborative work groups involved in mixed-motive tasks.

[1]  K. S. Raman,et al.  The Effect of GDSS and Elected Leadership on Small Group Meetings , 1991, J. Manag. Inf. Syst..

[2]  Starr Roxanne Hiltz,et al.  Distributed Group Support Systems , 1993, MIS Q..

[3]  Larry M. Coutts,et al.  Nonverbal Compensatory Reactions to Changes in Interpersonal Proximity , 1977 .

[4]  M. Nicholson.  The Resolution of Conflict , 1967 .

[5]  Alan R. Dennis,et al.  Media, Tasks, and Communication Processes: A Theory of Media Synchronicity , 2008, MIS Q..

[6]  Judith S. Olson,et al.  How a group-editor changes the character of a design meeting as well as its outcome , 1992, CSCW '92.

[7]  Guisseppi A. Forgionne,et al.  A multiple-criteria framework for evaluation of decision support systems , 2004 .

[8]  Starr Roxanne Hiltz,et al.  An Assessment of Group Support Systems Experimental Research: Methodology and Results , 1998, J. Manag. Inf. Syst..

[9]  Seungjin Whang,et al.  Analysis of interorganizational information sharing , 1993 .

[10]  Clarence A. Ellis,et al.  Nick Experimentation: Selected Results Concerning Effectiveness of Meeting Support Technology , 1989, J. Manag. Inf. Syst..

[11]  R. Banker.  Maximum likelihood, consistency and data envelopment analysis: a statistical foundation , 1993 .

[12]  Starr Roxanne Hiltz,et al.  Computer Support for Group Versus Individual Decisions , 1982, IEEE Trans. Commun..

[13]  Reza Barkhi.  The Effects of Decision Guidance and Problem Modeling on Group Decision-Making , 2002, J. Manag. Inf. Syst..

[14]  I. Benbasat,et al.  The Effects of Group, Task,Context, and Technology Variables on the Usefulness of Group Support Systems , 1993 .

[15]  Gerrit van Bruggen,et al.  DSS Effectiveness in Marketing Resource Allocation Decisions: Reality vs. Perception , 2004, Inf. Syst. Res..

[16]  Jay F. Nunamaker,et al.  Electronic meeting systems , 1991, CACM.

[17]  Ross Hightower,et al.  Effects of Communication Mode and Prediscussion Information Distribution Characteristics on Information Exchange in Groups , 1996, Inf. Syst. Res..

[18]  Ilze Zigurs,et al.  A Theory of Task/Technology Fit and Group Support Systems Effectiveness , 1998, MIS Q..

[19]  Rajiv D. Banker,et al.  A simulation study of hypothesis tests for differences in efficiencies , 1995 .

[20]  W. W. Muir,et al.  Regression Diagnostics: Identifying Influential Data and Sources of Collinearity , 1980 .

[21]  Peter R. Monge,et al.  The Dynamics of Organizational Proximity , 1985 .

[22]  D Byrne,et al.  Attraction toward the opposite sex as a determinant of physical proximity. , 1973, The Journal of social psychology.

[23]  Reza Barkhi,et al.  The influence of communication mode and incentive structure on GDSS process and outcomes , 2004, Decis. Support Syst..

[24]  Youngjin Yoo,et al.  Emergent leadership in virtual teams: what do emergent leaders do? , 2004, Inf. Organ..

[25]  José A. Pino,et al.  Groupware System Design and the Context Concept , 2004, CSCWD.

[26]  Gerardine DeSanctis,et al.  Computer-Based Support for Group Problem-Finding: An Experimental Investigation , 1988, MIS Q..

[27]  Nirup M. Menon,et al.  Information Technology Value through Different Normative Lenses , 2000, J. Manag. Inf. Syst..

[28]  Kent Marett,et al.  Deception: Toward an Individualistic View of Group Support Systems , 2008, J. Assoc. Inf. Syst..

[29]  J. Valacich,et al.  Group Support Systems: New Perspectives , 1992 .

[30]  John J. Sosik,et al.  Effects of Participative and Directive Leadership in Electronic Groups , 2004 .

[31]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[32]  A. Charnes,et al.  Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis , 1984 .

[33]  Milam W. Aiken,et al.  Group decision support systems and incentive structures , 1998, Inf. Manag..

[34]  Kwok Kee Wei,et al.  Interacting effects of GDSS and leadership , 1994, Decis. Support Syst..

[35]  Fred O. Walumbwa,et al.  Leadership: current theories, research, and future directions. , 2009, Annual review of psychology.

[36]  A. Charnes,et al.  Data Envelopment Analysis Theory, Methodology and Applications , 1995 .

[37]  Ramesh Sharda,et al.  Decision support system effectiveness: a review and an empirical test , 1988 .

[38]  Abraham Charnes,et al.  Measuring the efficiency of decision making units , 1978 .

[39]  David E. Gundersen.  Can DSS Technology Improve Group Decision Performance for End Users?: An Experimental Study , 1995 .

[40]  Rajiv D. Banker,et al.  Impact of Information Technology on Public Accounting Firm Productivity , 2002, J. Inf. Syst..

[41]  Edgar F. Borgatta,et al.  Some findings relevant to the great man theory of leadership. , 1954 .

[42]  H. Raghav Rao,et al.  Efficiency of critical incident management systems: Instrument development and validation , 2007, Decis. Support Syst..

[43]  G. A. Forgionne.  An AHP model of DSS effectiveness , 1999 .

[44]  Gerardine DeSanctis,et al.  A foundation for the study of group decision support systems , 1987 .

[45]  Michael H. Zack,et al.  Interactivity and Communication Mode Choice in Ongoing Management Groups , 1993, Inf. Syst. Res..

[46]  William W. Cooper,et al.  Evaluating Program and Managerial Efficiency: An Application of Data Envelopment Analysis to Program Follow Through , 1981 .

[47]  Robert J. Kauffman,et al.  Measuring Gains in Operational Efficiency from Information Technology: A Study of the Positran Deployment at Hardee'S Inc. , 1990, J. Manag. Inf. Syst..

[48]  R. Daft,et al.  Media Symbolism, Media Richness, and Media Choice in Organizations , 1987 .

[49]  Salvatore T. March,et al.  Design and natural science research on information technology , 1995, Decis. Support Syst..

[50]  Richard L. Daft,et al.  Organizational information requirements, media richness and structural design , 1986 .

[51]  G. Yukl,et al.  Leadership in Organizations , 1981 .

[52]  Moez Limayem,et al.  Impact of GDSS: Opening the black box , 2006, Decis. Support Syst..

[53]  Murray Turoff,et al.  Coordination Structures and System Restrictiveness in Distributed Group Support Systems , 2002 .

[54]  Rajiv D. Banker,et al.  A model to evaluate variables impacting the productivity of software maintenance projects , 1991 .

[55]  Reza Barkhi,et al.  A study of the effect of communication channel and authority on group decision processes and outcomes , 1998, Decis. Support Syst..

[56]  Wade D. Cook,et al.  Implementations of robotics: Identifying efficient implementors , 1992 .

[57]  Andrew B. Whinston,et al.  Incentives and Computing Systems for Team-Based Organizations , 1995 .

[58]  Starr Roxanne Hiltz,et al.  Group Decision Support: The Effects of Designated Human Leaders and Statistical Feedback in Computerized Conferences , 1991, J. Manag. Inf. Syst..

[59]  Simon French,et al.  Design and evaluation of an intelligent decision support system for nuclear emergencies , 2005, Decis. Support Syst..

[60]  Rajiv D. Banker,et al.  Evaluating Contextual Variables Affecting Productivity Using Data Envelopment Analysis , 2008, Oper. Res..