Think twice and then: combining or choosing in dialectical bootstrapping?

Individuals can partly recreate the "wisdom of crowds" within their own minds by combining nonredundant estimates they themselves have generated. Herzog and Hertwig (2009) showed that this accuracy gain could be boosted by urging people to actively think differently when generating a 2nd estimate ("dialectical bootstrapping"). Although the "crowd within" promises accuracy gains, it remains unclear whether and when people spontaneously reap those gains. What makes people combine their estimates rather than trying to identify the better one? This research found that participants were more likely to combine when they were instructed to actively contradict themselves. Furthermore, they were more likely to combine as the size of the disagreement between 1st and 2nd estimate grew. People thus acted as if they were hedging against the risk of making large errors. Finally, when people pursued a strategy other than combination, they were not able to outperform their crowd within.

[1]  J. Kruschke Bayesian estimation supersedes the t test. , 2013, Journal of experimental psychology. General.

[2]  Stefan M. Herzog,et al.  The Wisdom of Many in One Mind , 2009, Psychological science.

[3]  Johannes Müller-Trede Repeated judgment sampling: Boundaries , 2011, Judgment and Decision Making.

[4]  E. Wagenmakers A practical solution to the pervasive problems ofp values , 2007, Psychonomic bulletin & review.

[5]  H. Pashler,et al.  Measuring the Crowd Within , 2008, Psychological science.

[6]  Francis Tuerlinckx,et al.  Measuring the crowd within again: a pre-registered replication study , 2013, Front. Psychol..

[7]  J. Stroop Is the judgment of the group better than that of the average member of the group , 1932 .

[8]  R. Hertwig Tapping into the Wisdom of the Crowd—with Confidence , 2012, Science.

[9]  Stefan M. Herzog,et al.  The Crowd Within and the Benefits of Dialectical Bootstrapping , 2012, Psychological science.

[10]  A. Benjamin,et al.  Smaller is better (when sampling from the crowd within): Low memory-span individuals benefit more from multiple opportunities for estimation. , 2010, Journal of experimental psychology. Learning, memory, and cognition.

[11]  Richard P. Larrick,et al.  Strategies for revising judgment: how (and how well) people use others' opinions. , 2009, Journal of experimental psychology. Learning, memory, and cognition.

[12]  William A. Link,et al.  On thinning of chains in MCMC , 2012 .

[13]  H Gu,et al.  The effects of averaging subjective probability estimates between and within judges. , 2000, Journal of experimental psychology. Applied.

[14]  Richard P. Larrick,et al.  Intuitions About Combining Opinions: Misappreciation of the Averaging Principle , 2006, Manag. Sci..

[15]  Martyn Plummer,et al.  JAGS Version 3.3.0 user manual , 2012 .

[16]  Ilan Yaniv,et al.  Using advice from multiple sources to revise and improve judgments , 2007 .

[17]  Max Berniker,et al.  Bayesian approaches to sensory integration for motor control. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[18]  Nigel Harvey,et al.  Effects of judges' forecasting on their later combination of forecasts for the same outcomes , 2004 .

[19]  Jan Lorenz,et al.  The wisdom of crowds in one mind: How individuals can simulate the knowledge of diverse societies to reach better decisions , 2011 .

[20]  M. Landy,et al.  Weighted linear cue combination with possibly correlated error , 2003, Vision Research.

[21]  Albert E. Mannes,et al.  Judgmental aggregation strategies depend on whether the self is involved , 2011 .

[22]  Richard P. Larrick,et al.  The social psychology of the wisdom of crowds. , 2012 .

[23]  Robert L. Winkler,et al.  Multiple Experts vs. Multiple Methods: Combining Correlation Assessments , 2004, Decis. Anal..

[24]  M. Landy,et al.  Ideal-Observer Models of Cue Integration , 2012 .

[25]  T. Evgeniou,et al.  To combine or not to combine: selecting among forecasts and their combinations , 2005 .

[26]  Jarrod Had MCMC Methods for Multi-Response Generalized Linear Mixed Models: The MCMCglmm R Package , 2010 .

[27]  Ilan Yaniv,et al.  Receiving Other People's Advice: Influence and Benefit , 2004 .

[28]  J. Kruschke Doing Bayesian Data Analysis: A Tutorial with R and BUGS , 2010 .

[29]  Z. Dienes Bayesian Versus Orthodox Statistics: Which Side Are You On? , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[30]  A. Koriat,et al.  When Are Two Heads Better than One and Why? , 2012, Science.

[31]  H. Bülthoff,et al.  Merging the senses into a robust percept , 2004, Trends in Cognitive Sciences.

[32]  J. Kruschke What to believe: Bayesian methods for data analysis , 2010, Trends in Cognitive Sciences.

[33]  M. Lepper,et al.  Considering the opposite: a corrective strategy for social judgment. , 1984, Journal of personality and social psychology.

[34]  Johanna Peetz,et al.  The Planning Fallacy , 2010 .

[35]  G. Rees,et al.  The Neural Bases of Multistable Perception , 2022 .

[36]  R. Dawes Judgment under uncertainty: The robust beauty of improper linear models in decision making , 1979 .

[37]  J. Antonakis,et al.  Quantifying Accuracy Improvement in Sets of Pooled Judgments , 2013, Psychological science.

[38]  S. Bonaccio,et al.  Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences , 2006 .

[39]  W. Heath The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies , 2008 .

[40]  Jack B. Soll Intuitive Theories of Information: Beliefs about the Value of Redundancy , 1999, Cognitive Psychology.

[41]  John K Kruschke,et al.  Bayesian Assessment of Null Values Via Parameter Estimation and Model Comparison , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[42]  R. Baayen,et al.  Mixed-effects modeling with crossed random effects for subjects and items , 2008 .

[43]  Yaniv,et al.  Advice Taking in Decision Making: Egocentric Discounting and Reputation Formation. , 2000, Organizational behavior and human decision processes.

[44]  J. Armstrong,et al.  PRINCIPLES OF FORECASTING 1 Principles of Forecasting : A Handbook for Researchers and Practitioners , 2006 .

[45]  Andrew D. Martin,et al.  MCMCpack: Markov chain Monte Carlo in R , 2011 .

[46]  L. Ross,et al.  Naïve realism and capturing the “wisdom of dyads” , 2012 .