Overconfidence in Judgmental Forecasting

Overconfidence is a common finding in the forecasting research literature. Judgmental overconfidence leads people (1) to neglect decision aids, (2) to make predictions contrary to the base rate, and (3) to succumb to “groupthink.” To counteract overconfidence forecasters should heed six principles: (1) Consider alternatives, especially in new situations; (2) List reasons why the forecast might be wrong; (3) In group interaction, appoint a devil’s advocate; (4) Make an explicit prediction and then obtain feedback; (5) Treat the feedback you receive as valuable information; (6) When possible, conduct experiments to test prediction strategies. These principles can help people to avoid generating only reasons that bolster their predictions and to learn optimally by comparing a documented prediction with outcome feedback.

[1]  B. Fischhoff,et al.  On the Psychology of Experimental Surprises. , 1977 .

[2]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[3]  B. Fischhoff,et al.  Subjective Confidence in Forecasts. , 1982 .

[4]  S. Tomkins.  Computer simulation of personality , 1963 .

[5]  A. H. Murphy,et al.  Probability Forecasting in Meteorology , 1984 .

[6]  Barry M. Staw,et al.  Expo 86: An Escalation Prototype. , 1986 .

[7]  Janet A. Sniezek,et al.  Perceived Expertise and Its Effect on Confidence , 1994 .

[8]  S. Plous,et al.  A comparison of strategies for reducing interval overconfidence in group judgments , 1995 .

[9]  Buck K.W. Pei,et al.  The Effects of Decision Consequences on Auditors' Reliance on Decision Aids in Audit Planning☆☆☆ , 1997 .

[10]  M. O'Connor,et al.  An examination of the accuracy of judgemental confidence intervals in time series forecasting , 1989 .

[11]  M. Potter,et al.  Interference in Visual Recognition , 1964, Science.

[12]  David L. Ronis,et al.  Probability judgment accuracy: China, Japan, and the United States , 1989 .

[13]  Spyros Makridakis,et al.  Factors affecting judgmental forecasts and confidence intervals , 1989 .

[14]  Max H. Bazerman,et al.  Perspectives for Understanding Negotiation , 1985 .

[15]  Derek J. Koehler,et al.  Explanation, imagination, and confidence in judgment. , 1991, Psychological bulletin.

[16]  S. Oskamp.  OVERCONFIDENCE IN CASE-STUDY JUDGMENTS. , 1965, Journal of consulting psychology.

[17]  Janet A. Sniezek,et al.  Influences on the appropriateness of confidence in judgment: Practice, effort, information, and decision-making , 1991 .

[18]  R. Cialdini,et al.  Self-Relevant Scenarios as Mediators of Likelihood Estimates and Compliance: Does Imagining Make It So? , 1982 .

[19]  H. Arkes,et al.  Factors influencing the use of a decision rule in a probabilistic task , 1986 .

[20]  Kenneth R. Hammond,et al.  Judgment in a dynamic task: Microburst forecasting , 1991 .

[21]  Rebecca A. Henry,et al.  Accuracy and confidence in group judgment , 1989 .

[22]  Janet A. Sniezek,et al.  An examination of group process in judgmental forecasting , 1989 .

[23]  R. Devaul,et al.  Medical school performance of initially rejected students. , 1987, JAMA.

[24]  L. Ross,et al.  The role of construal processes in overconfident predictions about the self and others. , 1990, Journal of personality and social psychology.

[25]  Richard Gonzalez,et al.  Interaction with Others Increases Decision Confidence but Not Decision Quality: Evidence against Information Collection Views of Interactive Decision Making , 1995 .

[26]  D. Schweiger,et al.  Group Approaches for Improving Strategic Decision Making: A Comparative Analysis of Dialectical Inquiry, Devil's Advocacy, and Consensus , 1986 .

[27]  J. Keith Murnighan,et al.  Group Confidence Pressures in Iterative Decisions , 1982 .

[28]  Jay J.J. Christensen-Szalanski,et al.  Physicians' use of probabilistic information in a real clinical setting. , 1981 .

[29]  J. Yesavage,et al.  Reliability, accuracy, and decision-making strategy in clinical predictions of imminent dangerousness. , 1983, Journal of consulting and clinical psychology.

[30]  Charles R. Schwenk,et al.  Effects of the expert, devil's advocate, and dialectical inquiry methods on prediction performance , 1980 .

[31]  A. Tversky,et al.  The simulation heuristic , 1982 .

[32]  Stephen J. Hoch,et al.  Outcome feedback: hindsight and information , 1989 .

[33]  B. Fischhoff,et al.  Hindsight ≠ foresight: the effect of outcome knowledge on judgment under uncertainty* , 2003 .

[34]  Keith D. Markman,et al.  Multiple explanation: A consider-an-alternative strategy for debiasing judgments. , 1995 .

[35]  Ilan Yaniv,et al.  A case study of expert judgment: Economists' probabilities versus base-rate model forecasts , 1992 .

[36]  Michael O'Driscoll,et al.  Factors Influencing Wishful Thinking and Predictions of Election Outcomes , 1992 .

[37]  Janet A. Sniezek,et al.  Groups under uncertainty: An examination of confidence in group decision making☆ , 1992 .

[38]  B. Fischhoff,et al.  Calibration of probabilities: the state of the art to 1980 , 1982 .

[39]  Giuseppe Mancini,et al.  Intelligent Decision Support in Process Environments , 1986, NATO ASI Series.

[40]  B. Carlson,et al.  The Accuracy of Future Forecasts and Past Judgments , 1993 .

[41]  G. Gigerenzer,et al.  Probabilistic mental models: a Brunswikian theory of confidence. , 1991, Psychological review.

[42]  David L. Ronis,et al.  Components of probability judgment accuracy: Individual consistency and effects of subject matter and assessment method. , 1987 .

[43]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[44]  H. Arkes,et al.  Impediments to accurate clinical judgment and possible ways to minimize their impact. , 1981, Journal of consulting and clinical psychology.

[45]  D. Davis,et al.  New Projects : Beware of False Economies : Harvard Business Review , 1987 .

[46]  P. Wason.  On the Failure to Eliminate Hypotheses in a Conceptual Task , 1960 .

[47]  N V Dawson,et al.  Hemodynamic assessment in managing the critically ill: is physician confidence warranted? , 1993, Medical decision making : an international journal of the Society for Medical Decision Making.

[48]  Daniel Kahneman,et al.  Availability: A heuristic for judging frequency and probability , 1973 .

[49]  I. Janis.  Victims Of Groupthink , 1972 .

[50]  Philip E. Tetlock,et al.  Theory-Driven Reasoning About Plausible Pasts and Probable Futures in World Politics: Are We Prisoners of Our Preconceptions? , 1999 .

[51]  Fred D. Davis,et al.  Harmful effects of seemingly helpful information on forecasts of stock earnings , 1994 .

[52]  B. Fischhoff,et al.  Hindsight is not equal to foresight: The effect of outcome knowledge on judgment under uncertainty. , 1975 .

[53]  Steven J. Sherman,et al.  Social explanation: The role of timing, set, and recall on subjective likelihood estimates. , 1983 .

[54]  R. Buehler,et al.  Exploring the "planning fallacy": Why people underestimate their task completion times. , 1994 .

[55]  P. Juslin.  The Overconfidence Phenomenon as a Consequence of Informal Experimenter-Guided Selection of Almanac Items , 1994 .

[56]  J. K. Esser,et al.  Groupthink and the space shuttle Challenger accident: Toward a quantitative case analysis. , 1989 .

[57]  Stephen J. Hoch,et al.  Counterfactual reasoning and accuracy in predicting personal events. , 1985 .

[58]  R. Hogarth,et al.  Confidence in judgment: Persistence of the illusion of validity. , 1978 .

[59]  Nigel Harvey,et al.  Effects of difficulty on judgemental probability forecasting of control response efficacy , 1990 .

[60]  Lyle Brenner,et al.  Overconfidence in Probability and Frequency Judgments: A Critical Examination , 1996 .

[61]  David V. Budescu,et al.  On the Importance of Random Error in the Study of Probability Judgment. Part II: Applying the Stochastic Judgment Model to Detect Systematic Trends , 1997 .

[62]  L. Ross,et al.  The overconfidence effect in social prediction. , 1990, Journal of personality and social psychology.

[63]  Gideon Keren,et al.  Facing uncertainty in the game of bridge: A calibration study , 1987 .

[64]  W A Wagenaar,et al.  Does the expert know? The reliability of predictions and confidence ratings of experts , 1986 .

[65]  Whitecotton,et al.  Improving Predictive Accuracy with a Combination of Human Intuition and Mechanical Decision Aids. , 1998, Organizational behavior and human decision processes.

[66]  Max H. Bazerman,et al.  Improving negotiation effectiveness under final offer arbitration: The role of selection and training. , 1982 .

[67]  I. Erev,et al.  Simultaneous Over- and Underconfidence: The Role of Error in Judgment Processes. , 1994 .

[68]  Phillip E. Pfeifer,et al.  Are We Overconfident in the Belief That Probability Forecasters Are Overconfident , 1994 .

[69]  B. Fischhoff,et al.  Journal of Experimental Psychology: Human Learning and Memory , 1980 .

[70]  L. Ross,et al.  Overconfident prediction of future actions and outcomes by self and others. , 1990, Journal of personality and social psychology.

[71]  Anders Winman,et al.  Calibration of sensory and cognitive judgments : Two different accounts , 1993 .

[72]  H. Arkes,et al.  Two methods of reducing overconfidence , 1987 .

[73]  R. Bjork,et al.  Learning, Remembering, Believing: Enhancing Human Performance , 1994 .