Exploring the Relationship Between Topic Area Knowledge and Forecasting Performance

The Intelligence Community (IC) is often asked to make predictions about future world events. One aspect of predicting the quality of forecasts and forecasters is the knowledge that the forecaster has about the question to be forecast. This paper explores the relationship between factual knowledge about a forecast event and eventual performance on a forecast question. The results demonstrated a significant relationship between a forecaster answering a series of factual questions correctly and answering the corresponding forecast question correctly. This relationship is enhanced when controlling for the relative difficulty of the factual question. When controlling for forecaster performance, roughly half of the impact was due to general forecaster performance and half was due to their specific knowledge about a given forecast question. Interestingly, we found that forecasters with more factual knowledge were less calibrated with respect to their probability forecast whereas forecasters who were less knowledgeable were better calibrated in their probability estimates. We discuss the implication of the results related to improving forecast quality.