Group Decisions based on Confidence Weighted Majority Voting

Background: It has repeatedly been reported that when making decisions under uncertainty, groups outperform individuals. In a lab setting, real groups are often replaced by simulated groups: Instead of performing an actual group discussion, individual responses are aggregated by a numerical computation. While studies typically use unweighted majority voting (MV) for this aggregation, the theoretically optimal method is confidence weighted majority voting (CWMV) - if confidence ratings for the individual responses are available. However, it is not entirely clear how well the theoretically derived CWMV method predicts real group decisions and confidences. Therefore, we compared simulated group responses using CWMV and MV to real group responses. Results: Simulated group decisions based on CWMV matched the accuracy of real group decisions very well, while simulated group decisions based on MV showed lower accuracy. Also, CWMV well predicted the confidence that groups put into their group decisions. Yet, individuals and real groups showed a bias towards under--confidence while CWMV does not. Conclusion: Our results highlight the importance of taking into account individual confidences when investigating group decisions: We found that real groups can aggregate individual confidences such that they match the optimal aggregation given by CWMV. This implies that research using simulated group decisions should use CWMV and not MV.

[1]  A. Martins.  Probability biases as Bayesian inference , 2006, Judgment and Decision Making.

[2]  Ralf H. J. M. Kurvers,et al.  Collective Intelligence Meets Medical Decision-Making: The Collective Outperforms the Best Radiologist , 2015, PloS one.

[3]  M. Galesic,et al.  Smaller crowds outperform larger crowds and individuals in realistic task conditions. , 2016 .

[4]  G. Owen,et al.  Thirteen theorems in search of the truth , 1983 .

[5]  Reid Hastie,et al.  The robust beauty of majority rules in group decisions. , 2005, Psychological review.

[6]  Z. Dienes,et al.  Gambling on the unconscious: A comparison of wagering and confidence ratings as measures of awareness in an artificial grammar task , 2010, Consciousness and Cognition.

[7]  Shmuel Nitzan,et al.  Optimal Decision Rules in Uncertain Dichotomous Choice Situations , 1982 .

[8]  Nicholas Epley,et al.  Group discussion improves lie detection , 2015, Proceedings of the National Academy of Sciences.

[9]  S. Kaniovski,et al.  Optimal jury design for homogeneous juries with correlated votes , 2011 .

[10]  R D Sorkin,et al.  Signal-detection analysis of group decision making. , 2001, Psychological review.

[11]  Wolfgang Gaissmaier,et al.  Diagnostic performance by medical students working individually or in teams. , 2015, JAMA.

[12]  P. Fox,et al.  Confidence in Individual and Group Decision Making: When "Two Heads" Are Worse Than One. , 2004 .

[13]  M. Birnbaum,et al.  Testing a class of models that includes majority rule and regret theories: Transitivity, recycling, and restricted branch independence. , 2015 .

[14]  P. Latham,et al.  References and Notes Supporting Online Material Materials and Methods Figs. S1 to S11 References Movie S1 Optimally Interacting Minds R�ports , 2022 .

[15]  Janet A. Sniezek,et al.  Revision, Weighting, and commitment in consensus group judgment , 1990 .

[16]  Dan Bang,et al.  Making better decisions in groups , 2017, Royal Society Open Science.

[17]  J. Sonnemans,et al.  Judicial Error by Groups and Individuals , 2012 .

[18]  Jennifer A Kowalkowski.  Confidence in Measurement. , 2018, The Journal of rural health : official journal of the American Rural Health Association and the National Rural Health Care Association.

[19]  L. Shapley,et al.  Optimizing group judgmental accuracy in the presence of interdependencies , 1984 .

[20]  Mark A. Burgman,et al.  The collective intelligence of random small crowds: A partial replication of Kosinski et al. (2012) , 2019, Judgment and Decision Making.

[21]  Gjergji Kasneci,et al.  Crowd IQ: aggregating opinions to boost performance , 2012, AAMAS.

[22]  Lyle Brenner,et al.  Overconfidence in Probability and Frequency Judgments: A Critical Examination , 1996 .

[23]  Rebecca A. Henry,et al.  Accuracy and confidence in group judgment , 1989 .

[24]  Varun Gauria,et al.  Organizational Behavior and Human Decision Processes , 2019 .

[25]  A. Tversky,et al.  The weighing of evidence and the determinants of confidence , 1992, Cognitive Psychology.

[26]  Clintin P Davis-Stober,et al.  QTest: Quantitative Testing of Theories of Binary Choice. , 2014, Decisions.

[27]  Sébastien Massoni,et al.  Confidence measurement in the light of signal detection theory , 2014, Front. Psychol..

[28]  Stefan M. Herzog,et al.  Boosting medical diagnostics by pooling independent judgments , 2016, Proceedings of the National Academy of Sciences.

[29]  H. Mercier.  The Argumentative Theory: Predictions and Empirical Evidence , 2016, Trends in Cognitive Sciences.

[30]  Gjergji Kasneci,et al.  Crowd IQ: measuring the intelligence of crowdsourcing platforms , 2012, WebSci '12.

[31]  R. Dolan,et al.  Metacognition: computation, biology and function , 2012, Philosophical Transactions of the Royal Society B: Biological Sciences.