ConsensUs: Supporting Multi-Criteria Group Decisions by Visualizing Points of Disagreement

Groups often face difficulty reaching consensus. For complex decisions with multiple criteria, verbal and written discourse alone may impede groups from pinpointing and moving past fundamental disagreements. To help support consensus building, we introduce ConsensUs, a novel visualization tool that highlights disagreement by asking group members to quantify their subjective opinions across multiple criteria. To evaluate this approach, we conducted a between-subjects experiment with 87 participants on a comparative hiring task. The study compared three modes of sensemaking on a group decision: written discourse only, visualization only, and written discourse plus visualization. We confirmed that the visualization helped participants identify disagreements within the group and then measured subsequent changes to their individual opinions. The results show that disagreement highlighting led participants to align their ratings more with the opinions of other group members. While disagreement highlighting led to better score alignment, participants reported a number of reasons for shifting their score, from genuine consensus to appeasement. We discuss further research angles to understand how disagreement highlighting affects social processes and whether it produces objectively better decisions.

[1]  Paloma Díaz,et al.  Envisioning grid vulnerabilities: multi-dimensional visualization for electrical grid planning , 2012, AVI.

[2]  P. Wason.  On the Failure to Eliminate Hypotheses in a Conceptual Task , 1960 .

[3]  Katia Passerini,et al.  Seeing is believing (or at least changing your mind): The influence of visibility and task complexity on preference changes in computer‐supported team decision making , 2016, J. Assoc. Inf. Sci. Technol..

[4]  Suzanne D. Pawlowski,et al.  The Delphi method as a research tool: an example, design considerations and applications , 2004, Inf. Manag..

[5]  R. Nickerson.  Confirmation Bias: A Ubiquitous Phenomenon in Many Guises , 1998 .

[6]  Keng Siau,et al.  Strategic implications of mobile technology: A case study using Value-Focused Thinking , 2005, J. Strateg. Inf. Syst..

[7]  Raymond W. Preiss,et al.  Comparing the persuasiveness of narrative and statistical evidence using meta‐analysis , 1997 .

[8]  Daniel A. Keim,et al.  Pixel bar charts: a visualization technique for very large multi-attribute data sets? , 2002, Inf. Vis..

[9]  Antonio Tencati,et al.  The Collaborative Enterprise , 2009 .

[10]  Alexander Felfernig,et al.  Counteracting Anchoring Effects in Group Decision Making , 2015, UMAP.

[11]  L. Susskind,et al.  The consensus building handbook : a comprehensive guide to reaching agreement , 1999 .

[12]  Fabrizio Butera,et al.  Hidden Profiles and Concealed Information: Strategic Information Sharing and Use in Group Decision Making , 2009, Personality & social psychology bulletin.

[13]  S S Lam,et al.  Improving group decisions by better pooling information: a comparative advantage of group decision support systems. , 2000, The Journal of applied psychology.

[14]  S. R. Hiltz,et al.  Experiments in group decision making: Communication process and outcome in face-to-face versus computerized conferences. , 1986 .

[15]  Norman Miller,et al.  Ten years of research on the false-consensus effect: an empirical and theoretical review , 1987 .

[16]  L. Alberto Franco,et al.  Different paths to consensus? The impact of need for closure on model-supported group conflict management , 2016, Eur. J. Oper. Res..

[17]  Alan R. Dennis,et al.  Information exchange and use in GSS and verbal group decision making: effects of minority influence , 1997, Proceedings of the Thirtieth Hawaii International Conference on System Sciences.

[18]  Paul E. Jones,et al.  The potential for social contextual and group biases in team decision-making: biases, conditions and psychological mechanisms , 2000, Ergonomics.

[19]  Louis B. Rosenberg.  Human Swarms, a real-time method for collective intelligence , 2015, ECAL.

[20]  Alan Borning,et al.  Is this what you meant?: promoting listening on the web with reflect , 2012, CHI.

[21]  R. Scheines,et al.  Organizational Behavior and Human Decision Processes , 1977 .

[22]  Tommaso Turchi,et al.  Treemaps and the Visual Comparison of Hierarchical Multi-Attribute Data , 2016, AVI.

[23]  Susan G. Straus,et al.  ExpertLens: A System for Eliciting Opinions from a Large Pool of Non-Collocated Experts with Diverse Knowledge , 2011 .

[24]  Helmut Lamm,et al.  Group versus individual performance on tasks requiring ideational proficiency , 1973 .

[25]  Brian Mullen,et al.  The false consensus effect: A meta-analysis of 115 hypothesis tests , 1985 .

[26]  Hamed Taheri,et al.  Interactive visualization to facilitate group deliberations in decision making processes , 2015 .

[27]  Gennady L. Andrienko,et al.  Interactive visual tools for spatial multicriteria decision making , 2002, AVI '02.

[28]  J. Hackman,et al.  Interventions into group process: An approach to improving the effectiveness of groups , 1974 .

[29]  S. Weisband.  Group discussion and first advocacy effects in computer-mediated and face-to-face decision making groups , 1992 .

[30]  Robert O. Briggs,et al.  AIS Electronic Library (AISeL) , 2018 .

[31]  Ji Soo Yi.  Visualized decision making: development and application of information visualization techniques to improve decision quality of nursing home choice , 2008 .

[32]  Ivan Beschastnikh,et al.  Community, consensus, coercion, control: cs*w or how policy mediates mass participation , 2007, GROUP.

[33]  Anne P. Massey,et al.  New Product Development Decision-Making Effectiveness: Comparing Individuals, Face-To-Face Teams, and Virtual Teams , 2001, Decis. Sci..

[34]  Kenneth Y. Goldberg,et al.  Opinion space: a scalable tool for browsing online comments , 2010, CHI.

[35]  E.,et al.  GROUPS : INTERACTION AND PERFORMANCE , 2001 .

[36]  Susan R. Fussell,et al.  Effects of Sensemaking Translucence on Distributed Collaborative Analysis , 2016, CSCW.

[37]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[38]  K. Jehn.  A qualitative analysis of conflict types and dimensions in , 1997 .

[39]  H. Itakura.  Describing conversational dominance , 2001 .

[40]  K. Williams,et al.  Many Hands Make Light the Work: The Causes and Consequences of Social Loafing , 1979 .

[41]  Sara B. Kiesler,et al.  The Equalization Phenomenon: Status Effects in Computer-Mediated and Face-to-Face Decision-Making Groups , 1991, Hum. Comput. Interact..

[42]  Richard Chimera.  Value bars: an information visualization and navigation tool for multi-attribute listings , 1992, CHI '92.

[43]  Hanspeter Pfister,et al.  LineUp: Visual Analysis of Multi-Attribute Rankings , 2013, IEEE Transactions on Visualization and Computer Graphics.

[44]  Jeffrey Heer,et al.  Termite: visualization techniques for assessing textual topic models , 2012, AVI.

[45]  Sanjana Bajracharya,et al.  Interactive visualization for group decision-making , 2014 .

[46]  E. Krause,et al.  Taxicab Geometry: An Adventure in Non-Euclidean Geometry , 1987 .

[47]  Cristina Conati,et al.  Evaluating the Impact of User Characteristics and Different Layouts on an Interactive Visualization for Decision Making , 2014, Comput. Graph. Forum.

[48]  D. Winterfeldt,et al.  Cognitive and Motivational Biases in Decision and Risk Analysis , 2015, Risk analysis : an official publication of the Society for Risk Analysis.

[49]  D. Watson,et al.  Constructing validity: Basic issues in objective scale development , 1995 .

[50]  Francisco Herrera,et al.  A model of consensus in group decision making under linguistic assessments , 1996, Fuzzy Sets Syst..

[51]  Russell Mannion,et al.  Systematic biases in group decision-making: implications for patient safety. , 2014, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[52]  Alan Borning,et al.  Supporting reflective public thought with considerit , 2012, CSCW.

[53]  S. Kiesler,et al.  Group decision making and communication technology , 1992 .

[54]  P. Riehmann,et al.  Interactive Sankey diagrams , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[55]  Brian P. Bailey,et al.  Procid: Bridging Consensus Building Theory with the Practice of Distributed Design Discussions , 2015, CSCW.

[56]  I. Janis.  Victims of Groupthink: A psychological study of foreign-policy decisions and fiascoes. By Irving L. Janis. (Boston: Houghton Mifflin, 1972. viii + 276 pp. Map, illustrations, chart, notes, sources, bibliography, and index. Cloth, $7.95; paper $4.50.) , 1973 .

[57]  Mark Klein,et al.  Enabling Large-Scale Deliberation Using Attention-Mediation Metrics , 2011, Computer Supported Cooperative Work (CSCW).

[58]  B. Rosenberg.  Human Swarms , a real-time paradigm for Collective Intelligence , 2015 .

[59]  Susan R. Fussell,et al.  Do visualizations improve synchronous remote collaboration? , 2008, CHI.

[60]  Starr Roxanne Hiltz,et al.  A comparative content analysis of face-to-face vs. asynchronous group decision making , 2003, Decis. Support Syst..

[61]  B. Baltes,et al.  Computer-Mediated Communication and Group Decision Making: A Meta-Analysis , 2002 .

[62]  Murray Turoff,et al.  The Delphi Method: Techniques and Applications , 1976 .

[63]  J. Hahn.  Victims Of Groupthink A Psychological Study Of Foreign Policy Decisions And Fiascoes , 2016 .

[64]  Chen-Tung Chen,et al.  Extensions of the TOPSIS for group decision-making under fuzzy environment , 2000, Fuzzy Sets Syst..

[65]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[66]  Ralph L. Keeney,et al.  Value-Focused Thinking , 1996 .

[67]  Theodor J. Stewart,et al.  Multicriteria Decision Making , 1999 .

[68]  Francisco Herrera,et al.  Visualizing Consensus in Group Decision Making Situations , 2007, 2007 IEEE International Fuzzy Systems Conference.

[69]  Roland Castellanos.  A Qualitative Analysis of Conflict Types and Dimensions in Organizational Groups , 2007 .

[70]  T. Gal,et al.  Multicriteria Decision Making: Advances in MCDM Models, Algorithms, Theory, and Applications , 2012 .

[71]  Giuseppe Carenini,et al.  ValueCharts: analyzing linear models expressing preferences and evaluations , 2004, AVI.

[72]  D. Myers,et al.  The group polarization phenomenon. , 1976 .

[73]  Enrique Herrera-Viedma,et al.  Using Visualization Tools to Guide Consensus in Group Decision Making , 2007, WILF.

[74]  Mike Allen,et al.  Testing the persuasiveness of evidence: Combining narrative and statistical forms , 2000 .

[75]  Walter Bender,et al.  Changing small group interaction through visual reflections of social behavior , 2005 .

[76]  C. Sunstein,et al.  Deliberating Groups versus Prediction Markets (or Hayek's Challenge to Habermas) , 2006, Episteme.

[77]  A. Hollingshead.  The Rank-Order Effect in Group Decision Making , 1996 .

[78]  M. Sheelagh T. Carpendale,et al.  STRATOS: Using Visualization to Support Decisions in Strategic Software Release Planning , 2015, CHI.

[79]  Norman Crolee Dalkey,et al.  An experimental study of group opinion , 1969 .

[80]  Tobias Greitemeyer,et al.  Preference-consistent evaluation of information in the hidden profile paradigm: beyond group-level explanations for the dominance of shared information in group decisions. , 2003, Journal of personality and social psychology.

[81]  G. Stasser,et al.  Hidden Profiles: A Brief History , 2003 .

[82]  Shelly Farnham,et al.  Structured online interactions: improving the decision-making of small discussion groups , 2000, CSCW '00.

[83]  John T. Stasko,et al.  Toward a Deeper Understanding of the Role of Interaction in Information Visualization , 2007, IEEE Transactions on Visualization and Computer Graphics.