The recalcitrance of overconfidence and its contribution to decision aid neglect

Three experiments tested the hypothesis that people’s overconfidence in the quality of their intuitive judgment strategies contributes to their reluctance to use helpful actuarial judgment aids. Participants engaged in a judgment task that required them to use five cues to decide whether a prospective juror favored physician-assisted suicide. Participants had the opportunity to examine the judgments of a statistical equation that correctly classified 77% of the prospective jurors. In all experiments, participants infrequently examined the equation, performed worse than the equation, and were highly overconfident. In Experiments 1 and 2, outcome feedback and calibration feedback failed to reduce overconfidence. In Experiment 3, enhanced calibration feedback reduced overconfidence and increased reliance on the equation, thus leading to improved judgment performance. Copyright # 2005 John Wiley & Sons, Ltd.

[1]  J. Merenstein,et al.  Applying the acute ischemic heart disease predictive instrument. , 1987, The Journal of family practice.

[2]  R. Hogarth,et al.  Confidence in judgment: Persistence of the illusion of validity. , 1978 .

[3]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[4]  Elizabeth S. Veinott,et al.  Hard decisions, bad decisions: On decision quality and decision aiding , 2003 .

[5]  Janet A. Sniezek,et al.  The effect of choosing on confidence in choice , 1990 .

[6]  Winston R. Sieck,et al.  Cross-Cultural Variations in Probability Judgment Accuracy: Beyond General Knowledge Overconfidence? , 1998, Organizational behavior and human decision processes.

[7]  B. Fischhoff,et al.  Subjective Confidence in Forecasts. , 1982 .

[8]  Steven L. Wise,et al.  The Development and Validation of a Scale Measuring Attitudes toward Statistics , 1985 .

[9]  Winston R. Sieck,et al.  Overconfidence effects in category learning: a comparison of connectionist and exemplar memory models. , 2001, Journal of experimental psychology. Learning, memory, and cognition.

[10]  R. Zajonc.  Attitudinal effects of mere exposure. , 1968 .

[11]  A Alvarado,et al.  A practical score for the early diagnosis of acute appendicitis. , 1986, Annals of emergency medicine.

[12]  Derek J. Koehler,et al.  Heuristics and Biases: The Calibration of Expert Judgment: Heuristics and Biases Beyond the Laboratory , 2002 .

[13]  P. Slovic,et al.  An analysis-of-variance model for the assessment of configural cue utilization in clinical judgment. , 1968, Psychological bulletin.

[14]  Michael J. Renner,et al.  But I thought I knew that: using confidence estimation as a debiasing technique to improve classroom performance , 2001 .

[15]  J. Frank Yates,et al.  Subjective probability accuracy analysis. , 1994 .

[16]  Ilana Ritov,et al.  The affective costs of overconfidence , 2004 .

[17]  P. Meehl.  Causes and effects of my disturbing little book. , 1986, Journal of personality assessment.

[18]  M. Dougherty,et al.  Integration of the ecological and error models of overconfidence using a multiple-trace memory model. , 2001, Journal of experimental psychology. General.

[19]  D. Mcniel,et al.  Screening for risk of inpatient violence , 1994 .

[20]  B. Brehmer.  Cue utilization and cue consistency in multiple-cue probability learning☆ , 1972 .

[21]  Winston R. Sieck,et al.  Exposition Effects on Decision Making: Choice and Confidence in Choice , 1997 .

[22]  T. Rakow,et al.  Improving calibration without training: the role of task information , 2003 .

[23]  N. Schwarz,et al.  Accessibility experiences and the hindsight bias: I knew it all along versus it could never have happened , 2002, Memory & cognition.

[24]  Winston R. Sieck,et al.  Effects of Choice and Relative Frequency Elicitation on Overconfidence: Further Tests of an Exemplar-retrieval Model , 2003 .

[25]  H. Arkes,et al.  Factors influencing the use of a decision rule in a probabilistic task , 1986 .

[26]  Corey Ga,et al.  Applying the acute ischemic heart disease predictive instrument. , 1987 .

[27]  J. W. Hutchinson,et al.  Knowledge Calibration: What Consumers Know and What They Think They Know , 2000 .

[28]  G. Gigerenzer,et al.  Probabilistic mental models: a Brunswikian theory of confidence. , 1991, Psychological review.

[29]  Dane K. Peterson,et al.  Effect of input from a mechanical model on clinical judgment. , 1986 .

[30]  M. Howell,et al.  Awareness and use of the Ottawa ankle and knee rules in 5 countries: can publication alone be enough to change practice? , 2001, Annals of emergency medicine.

[31]  R. Buehler,et al.  Frequency, Probability, and Prediction: Easy Solutions to Cognitive Illusions? , 1999, Cognitive Psychology.

[32]  P. C. Price,et al.  Effects of a Relative-Frequency Elicitation Question on Likelihood Judgment Accuracy: The Case of External Correspondence. , 1998, Organizational behavior and human decision processes.

[33]  B. Brehmer.  In one word: Not from experience. , 1980 .

[34]  Kenneth R. Hammond,et al.  Negative effects of outcome-feedback in multiple-cue probability learning. , 1973 .

[35]  C. Gettys,et al.  MINERVA-DM: A memory processes model for judgments of likelihood. , 1999 .

[36]  L. Brenner,et al.  Should observed overconfidence be dismissed as a statistical artifact? Critique of Erev, Wallsten, and Budescu (1994) , 2000, Psychological review.

[37]  H. Arkes,et al.  Two methods of reducing overconfidence , 1987 .

[38]  J. Michael O'Malley,et al.  Learning Strategies in Second Language Acquisition: A cognitive theory of learning , 1990 .

[39]  William F. Brewer,et al.  The Role of Anomalous Data in Knowledge Acquisition: A Theoretical Framework and Implications for Science Instruction , 1993 .

[40]  F E Harrell,et al.  The Covariance Decomposition of the Probability Score and Its Use in Evaluating Prognostic Estimates , 1995, Medical decision making : an international journal of the Society for Medical Decision Making.

[41]  Winston R. Sieck,et al.  Heuristics and Biases: Probability Judgment across Cultures , 2002 .

[42]  Andrew C. Pollock,et al.  An application of probability judgement accuracy measures to currency forecasting , 1996 .

[43]  P Slovic,et al.  Cue-consistency and cue-utilization in judgment. , 1966, The American journal of psychology.

[44]  P. Slovic.  The Construction of Preference , 1995 .

[45]  R. Dawes.  Judgment under uncertainty: The robust beauty of improper linear models in decision making , 1979 .

[46]  P. Todd,et al.  Simple Heuristics That Make Us Smart , 1999 .

[47]  Cross-national variation in probability judgment , 1992 .

[48]  T. O. Nelson,et al.  A comparison of current measures of the accuracy of feeling-of-knowing predictions , 1984 .

[49]  P. Juslin,et al.  Naive empiricism and dogmatism in confidence research: a critical examination of the hard-easy effect. , 2000, Psychological review.

[50]  M. Levine.  A cognitive theory of learning: Research on hypothesis testing. , 1975 .

[51]  Baruch Fischhoff,et al.  A Little Learning...: Confidence in Multicue Judgment Tasks390664 , 1978 .

[52]  Ab McClelland,et al.  The calibration of subjective probabilities: Theories and models 1980-1993 , 1994 .

[53]  F. Craik,et al.  Levels of Pro-cessing: A Framework for Memory Research , 1975 .

[54]  Kenneth S. Shultz,et al.  Evidence of Reliability and Validity for Wise's Attitude toward Statistics Scale , 1998 .

[55]  B. Fischhoff,et al.  Training for calibration. , 1980 .

[56]  David V. Budescu,et al.  On the Importance of Random Error in the Study of Probability Judgment. Part II: Applying the Stochastic Judgment Model to Detect Systematic Trends , 1997 .

[57]  Gideon Keren,et al.  Facing uncertainty in the game of bridge: A calibration study , 1987 .

[58]  P. Juslin,et al.  Visual perception of dynamic properties: cue heuristics versus direct-perceptual competence. , 2000, Psychological review.

[59]  A. Tversky,et al.  On the psychology of prediction , 1973 .

[60]  G. Brier,et al.  External correspondence: Decompositions of the mean probability score , 1982 .

[61]  Ido Erev,et al.  On the Importance of Random Error in the Study of Probability Judgment. Part I: New Theoretical Developments , 1997 .

[62]  Robert H. Ashton,et al.  Pressure And Performance In Accounting Decision Settings - Paradoxical Effects Of Incentives, Feedback, And Justification , 1990 .

[63]  G. Gigerenzer.  How to Make Cognitive Illusions Disappear: Beyond “Heuristics and Biases” , 1991 .