Are We Wise About the Wisdom of Crowds? The Use of Group Judgments in Belief Revision

Recent research has advanced our understanding of how people use the advice of others to update their beliefs. Because groups and teams play a significant role in organizations and collectively are wiser than their individual members, it is important to understand their influence on belief revision as well. I report the results of four studies examining intuitions about group wisdom and the informational influence of groups. In their overt assessments, experimental participants rated larger groups as more accurate than smaller groups and discriminated more between them when group size was salient. When provided advice, participants relied more on groups than individuals to update their beliefs, but were only modestly sensitive to group size. Most were suboptimal in the use of that advice, overweighting their initial beliefs and underweighting the more valid judgment of the group. Thus although acknowledged in principle, the wisdom of crowds is only shallowly manifest in observed behavior.

[1]  J. Campbell,et al.  Informational and normative routes to conformity: The effect of faction size as a function of norm extremity and attention to the stimulus. , 1989 .

[2]  R. Hastie,et al.  Proper analysis of the accuracy of group judgments , 1997 .

[3]  Steven D. Penrod,et al.  Social Influence Model: A formal integration of research on majority and minority influence processes. , 1984 .

[4]  N. Harvey,et al.  Taking Advice: Accepting Help, Improving Judgment, and Sharing Responsibility☆☆☆ , 1997 .

[5]  Muzafer Sherif,et al.  A study of some social factors in perception. , 1935 .

[6]  James Surowiecki.  The wisdom of crowds: Why the many are smarter than the few and how collective wisdom shapes business, economies, societies, and nations Doubleday Books. , 2004 .

[7]  R. Bond,et al.  Group Size and Conformity , 2005 .

[8]  Adrian K. Rantilla,et al.  The effects of asymmetry among advisors on the aggregation of their opinions , 2003 .

[9]  Richard P. Larrick,et al.  Intuitions About Combining Opinions: Misappreciation of the Averaging Principle , 2006, Manag. Sci..

[10]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[11]  J. P. Morgan,et al.  Design and Analysis: A Researcher's Handbook , 2005, Technometrics.

[12]  Adrian K. Rantilla,et al.  Confidence in aggregation of expert opinions. , 2000, Acta psychologica.

[13]  Student.  BELIEF IN THE LAW OF SMALL NUMBERS , 1994 .

[14]  Nigel Harvey,et al.  Effects of judges' forecasting on their later combination of forecasts for the same outcomes , 2004 .

[15]  Janet A. Sniezek,et al.  Trust, Confidence, and Expertise in a Judge-Advisor System. , 2001, Organizational behavior and human decision processes.

[16]  Bernard Grofman,et al.  Information pooling and group decision making : proceedings of the Second University of California, Irvine, Conference on Political Economy , 1986 .

[17]  R. Hogarth,et al.  Order effects in belief updating: The belief-adjustment model , 1992, Cognitive Psychology.

[18]  B. Latané.  The psychology of social impact. , 1981 .

[19]  R. Thaler.  Toward a positive theory of consumer choice , 1980 .

[20]  H. Kelman.  Compliance, identification, and internalization three processes of attitude change , 1958 .

[21]  A. Jenness,et al.  The role of discussion in changing opinion regarding a matter of fact. , 1932 .

[22]  Christopher K. Hsee,et al.  Preference Reversals between Joint and Separate Evaluations of Options: A Review and Theoretical Analysis , 1999 .

[23]  V. L. Allen.  Situational Factors In Conformity1 , 1965 .

[24]  Nigel Harvey,et al.  Combining forecasts: What information do judges need to outperform the simple average? , 1999 .

[25]  Richard P. Larrick,et al.  Strategies for revising judgment: how (and how well) people use others' opinions. , 2009, Journal of experimental psychology. Learning, memory, and cognition.

[26]  Sanford L. Braver,et al.  Monotonic hypotheses in multiple group designs: A Monte Carlo study. , 1993 .

[27]  G. W. Hill.  Group versus individual performance: are n + 1 heads better than one?" psychological bulletin , 1982 .

[28]  Elliot Aronson,et al.  The handbook of social psychology, 2nd ed. , 1968 .

[29]  Ray W. Cooksey,et al.  Judgment analysis : theory, methods, and applications , 1996 .

[30]  Verlin B. Hinsz,et al.  The emerging conceptualization of groups as information processors. , 1997, Psychological bulletin.

[31]  Yaniv,et al.  Advice Taking in Decision Making: Egocentric Discounting and Reputation Formation. , 2000, Organizational behavior and human decision processes.

[32]  P. Hoffman.  The paramorphic representation of clinical judgment. , 1960, Psychological bulletin.

[33]  Michael Lawrence,et al.  Influence of differentiated roles on group forecasting accuracy , 2011 .

[34]  Ilan Yaniv,et al.  Using advice from multiple sources to revise and improve judgments , 2007 .

[35]  J. Scott Armstrong,et al.  Principles of forecasting , 2001 .

[36]  Ilan Yaniv.  ORGANIZATIONAL BEHAVIOR AND HUMAN DECISION PROCESSES, VOLUME 69, ISSUE 3 , 1997 .

[37]  E. Brunswik.  Representative design and probabilistic theory in a functional psychology. , 1955, Psychological review.

[38]  Klaus Fiedler,et al.  Information Sampling and Adaptive Cognition , 2005 .

[39]  Christopher K. Hsee,et al.  When Is More Better? , 2005 .

[40]  Ward Edwards,et al.  Judgment under uncertainty: Conservatism in human information processing , 1982 .

[41]  S. Milgram,et al.  LIBERATING EFFECTS OF GROUP PRESSURE. , 1965, Journal of personality and social psychology.

[42]  J. Ord,et al.  Principles of forecasting: A handbook for researchers and practitioners , 2002 .

[43]  R. Dawes,et al.  Linear models in decision making. , 1974 .

[44]  D. Kahneman,et al.  Does Living in California Make People Happy? A Focusing Illusion in Judgments of Life Satisfaction , 1998 .

[45]  R. Clemen.  Combining forecasts: A review and annotated bibliography , 1989 .

[46]  D. Gilbert,et al.  Focalism: a source of durability bias in affective forecasting. , 2000, Journal of personality and social psychology.

[47]  Derek J. Koehler,et al.  Illusion of confirmation from exposure to another's hypothesis , 2006 .

[48]  Robin M. Hogarth,et al.  Quality of Group Judgment , 1977 .

[49]  R. Hogarth,et al.  Unit weighting schemes for decision making , 1975 .

[50]  A. Tversky,et al.  Support theory: A nonextensional representation of subjective probability. , 1994 .

[51]  Klaus Fiedler,et al.  Confidence in Aggregation of Opinions from Multiple Sources , 2005 .

[52]  A. Tversky,et al.  Choices, Values, and Frames , 2000 .

[53]  G. Keppel,et al.  Design and Analysis: A Researcher's Handbook , 1976 .

[54]  Roger K. Blashfield,et al.  Performance of a composite as a function of the number of judges , 1978 .

[55]  Sabine A. Einwiller,et al.  Accuracy Motivation, Consensus Information, and the Law of Large Numbers: Effects on Attitude Judgment in the Absence of Argumentation , 1998 .

[56]  Robin M. Hogarth,et al.  A note on aggregating opinions , 1978 .

[57]  Fischer,et al.  Using Advice and Assessing Its Quality. , 2000, Organizational behavior and human decision processes.

[58]  Albert E. Mannes,et al.  Judgmental aggregation strategies depend on whether the self is involved , 2011 .

[59]  M. O'Connor,et al.  Judgemental adjustment of initial forecasts: Its effectiveness and biases , 1995 .

[60]  David A. Wilder,et al.  Perception of groups, size of opposition, and social influence. , 1977 .

[61]  L. Festinger.  A Theory of Social Comparison Processes , 1954 .

[62]  Ilan Yaniv,et al.  Receiving Other People's Advice: Influence and Benefit , 2004 .

[63]  S. Asch.  Effects of Group Pressure Upon the Modification and Distortion of Judgments , 1951 .

[64]  S. Goldberg,et al.  Three situational determinants of conformity to social norms. , 1954, Journal of abnormal psychology.

[65]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[66]  S. Bonaccio,et al.  Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences , 2006 .

[67]  M. Deutsch,et al.  A study of normative and informational social influences upon individual judgement. , 1955, Journal of abnormal psychology.

[68]  M. Birnbaum,et al.  Source Credibility in Social Judgment : Bias , Expertise , and the Judge ' s Point of View , 1979 .

[69]  B. Latané,et al.  The Social Impact of Majorities and Minorities. , 1981 .

[70]  Laureen A. Maines.  An experimental examination of subjective forecast combination , 1996 .

[71]  Dennis J. Devine,et al.  Teams in Organizations , 1999 .