Subjective Confidence in Forecasts.

Forecasts have little value to decision makers unless it is known how much confidence to place in them. Those expressions of confidence have, in turn, little value unless forecasters are able to assess the limits of their own knowledge accurately Previous research has shown very robust patterns in the judgements of individuals who have not received special training in confidence assessment: Knowledge generally increases as confidence increases. However, it increases too swiftly, with a doubling of confidence being associated with perhaps a 50 per cent increase in knowledge. With all but the easiest of tasks, people tend to be overconfident regarding how much they know These results have typically been derived from studies of judgements of general knowledge. The present study found that they also pertained to confidence in forecasts. Indeed, the confidence-knowledge curves observed here were strikingly similar to those observed previously. The only deviation was the discovery that a substantial minority of judges never expressed complete confidence in any of their forecasts. These individuals also proved to be better assessors of the extent of their own knowledge Apparently confidence in forecasts is determined by processes similar to those that determine confidence in general knowledge. Decision makers can use forecasters assessments in a relative sense, in order to predict when they are more and less likely to be correct. However, they should be hesitant to take confidence assessments literally. Someone is more likely to be right when he or she is ‘certain’than when he or she is ‘fairly confident’; but there is no guarantee that the certain forecast will come true.

[1]  P. A. Adams,et al.  Training in confidence-judgments. , 1958, The American journal of psychology.

[2]  Frank R. Clarke Confidence Ratings, Second‐Choice Responses, and Confusion Matrices in Intelligibility Tests , 1960 .

[3]  P. A. Adams,et al.  Realism of confidence judgments. , 1961, Psychological review.

[4]  Stuart Oskamp,et al.  The relationship of clinical experience and training methods to several criteria of clinical prediction. , 1962 .

[5]  Halbert E. Root,et al.  Probability Statements in Weather Forecasting , 1962 .

[6]  Raymond S. Nickerson,et al.  Confidence Ratings and Level of Performance on a Judgmental Task , 1965, Perceptual and motor skills.

[7]  C. Holstein An Experiment in Probabilistic Weather Forecasting , 1971 .

[8]  Carl-Axel S. Staël von Holstein,et al.  Probabilistic forecasting: An experiment related to the stock market , 1972 .

[9]  P. Slovic Psychological Study of Human Judgment: Implications for Investment Decision-Making , 1972 .

[10]  Ralf E. Schaefer,et al.  The assessment of subjective probability distributions: A training experiment☆ , 1973 .

[11]  Peter A. Morris,et al.  Decision Analysis Expert Use , 1974 .

[12]  J. B. Wallace,et al.  A STUDY OF THE PERFORMANCE OF SUBJECTIVE PROBABILITY ASSESSORS , 1974 .

[13]  A. H. Murphy,et al.  subjective probability forecasting experiments in meteorology: some preliminary results , 1974 .

[14]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[15]  B. Fischhoff,et al.  Hindsight is not equal to foresight: The effect of outcome knowledge on judgment under uncertainty. , 1975 .

[16]  B. Fischhoff,et al.  I knew it would happen: Remembered probabilities of once—future things , 1975 .

[17]  J. Dowie,et al.  On the Efficiency and Equity of Betting Markets , 1976 .

[18]  Baruch Fischhoff,et al.  The effect of temporal setting on likelihood estimates , 1976 .

[19]  S. Lichtenstein,et al.  Do those who know more also know more about how much they know?*1 , 1977 .

[20]  D H Gustafson,et al.  Comparison of five methods for estimating subjective probability distributions. , 1977, Organizational behavior and human performance.

[21]  B. Fischhoff,et al.  Knowing with Certainty: The Appropriateness of Extreme Confidence. , 1977 .

[22]  P. G. Moore,et al.  The Manager's Struggles with Uncertainty , 1977 .

[23]  Henry J. Steadman,et al.  PREDICTION IN PSYCHIATRY: AN EXAMPLE OF MISPLACED CONFIDENCE IN EXPERTS* , 1978 .

[24]  D. A. Seaver,et al.  Eliciting subjective probability distributions on continuous variables , 1978 .

[25]  Probability Estimates by Respondents: Does Weighting Improve Accuracy? , 1978 .

[26]  W. C. Howell,et al.  Uncertainty measurement: A cognitive taxonomy , 1978 .

[27]  K. Armelius,et al.  Task predictability and performance as determinants of confidence in multiple-cue judgments , 1979 .

[28]  J. R. Larson,et al.  The Equivalence Interval as a Measure of Uncertainty. , 1979 .

[29]  John G. Borkowski,et al.  Searching for Metamemory-Memory Connections: A Developmental Study , 1980 .

[30]  J. E. Selvidge,et al.  ASSESSING THE EXTREMES OF PROBABILITY DISTRIBUTIONS BY THE FRACTILE METHOD , 1980 .

[31]  B. Fischhoff,et al.  Training for calibration. , 1980 .

[32]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[33]  Spyros Makridakis,et al.  Forecasting and Planning: An Evaluation , 1981 .

[34]  Robert J. Genetski,et al.  Long-Range Forecasting: From Crystal Ball to Computer , 1981 .