A case study of expert judgment: Economists' probabilities versus base-rate model forecasts

In this case study of economists' forecasts concerning economic downturn, we examine key issues concerning the psychology of prediction and the controversy surrounding the value of expertise in forecasting. We examine when experts' knowledge promotes forecast accuracy and whether biases found in psychological studies (including underutilization of relevant base rates and tendencies to extreme prediction) occur in these economic forecasts. Experts' forecasts were compared to forecasts derived from base-rate models that relied on the historical frequencies of economic downturns. The performance patterns of the experts and models crossed over the forecast horizon. Experts outperformed models in shorter-term forecasting, whereas models outperformed experts in longer-term forecasting. These results highlight the abilities and limits of experts and models in prediction and the sources of their inaccuracy.

[1]  Y. Trope,et al.  The effects of base rates and individuating information on judgments about another person , 1980 .

[2]  A. Tversky,et al.  On the psychology of prediction , 1973 .

[3]  M. Chi,et al.  The Nature of Expertise , 1988 .

[4]  A. H. Murphy,et al.  A comparative evaluation of objective and subjective weather forecasts in the united states , 1984 .

[5]  Ilan Yaniv,et al.  Measures of Discrimination Skill in Probabilistic Judgment , 1991 .

[6]  Gerd Gigerenzer,et al.  Presentation and content: The use of base rates as a continuous variable. , 1988 .

[7]  B Kleinmuntz,et al.  Why we still use our heads instead of formulas: toward an integrative approach. , 1990, Psychological bulletin.

[8]  B. Fischhoff,et al.  Calibration of probabilities: the state of the art to 1980 , 1982 .

[9]  I. Ajzen.  Intuitive theories of events and the effects of base-rate information on prediction. , 1977 .

[10]  A. H. Murphy,et al.  “Good” Probability Assessors , 1968 .

[11]  R. Dawes.  Judgment under uncertainty: The robust beauty of improper linear models in decision making , 1979 .

[12]  Robert C. Blattberg,et al.  Database Models And Managerial Intuition: 50% Model + 50% Manager , 1990 .

[13]  A. H. Murphy.  A New Vector Partition of the Probability Score , 1973 .

[14]  W. Wagenaar,et al.  Temporal aspects of probabilistic predictions , 1987 .

[15]  Victor Zarnowitz,et al.  Rational Expectations and Macroeconomic Forecasts , 1985 .

[16]  Stephen K. McNees.  The role of judgment in macroeconomic forecasting accuracy , 1990 .

[17]  Shawn P. Curley,et al.  Conditional distribution analyses of probabilistic forecasts , 1985 .

[18]  Stuart Oskamp.  Judgment under uncertainty: Overconfidence in case-study judgments , 1982 .

[19]  Paul E. Meehl,et al.  Clinical Versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence , 1996 .

[20]  Robert J. Genetski,et al.  Long-Range Forecasting: From Crystal Ball to Computer , 1981 .

[21]  Hillel J. Einhorn,et al.  Expert measurement and mechanical combination , 1972 .

[22]  John B. Kidd,et al.  The utilization of subjective probabilities in production planning , 1970 .

[23]  David V. Budescu,et al.  Encoding subjective probabilities: A psychological and psychometric review , 1983 .

[24]  R. Hogarth,et al.  Confidence in judgment: Persistence of the illusion of validity. , 1978 .

[25]  T. Wallsten.  Physician and Medical Student Bias in Evaluating Diagnostic Information , 1981, Medical decision making : an international journal of the Society for Medical Decision Making.

[26]  W. Knaus,et al.  Short-term mortality predictions for critically ill hospitalized adults: science and ethics. , 1991, Science.

[27]  Robin M. Hogarth,et al.  Judgmental Versus Statistical Prediction: Information Asymmetry and Combination Rules , 1993 .