Combining Forecasts: Evidence on the relative accuracy of the simple average and Bayesian Model Averaging for predicting social science problems

The present study shows that the predictive performance of Ensemble Bayesian Model Averaging (EBMA) strongly depends on the conditions of the forecasting problem. EBMA is of limited value when uncertainty is high, a situation that is common for social science problems. In such situations, one should avoid methods that bear the risk of overfitting. Instead, one should acknowledge the uncertainty in the environment and use conservative methods that are robust when predicting new data. When combining forecasts, consider calculating simple (unweighted) averages of the component forecasts. A vast prior literature finds that simple averages yield forecasts that are often at least as accurate as those from more complex combining methods. These results also hold for the use of EBMA for social science problems. A summary of results from the domain of economic forecasting shows that the simple average was more accurate than EBMA in three out of four studies. On average, the error of the EBMA forecasts was 6% higher than the error of the simple average. A reanalysis and extension of a published study, which had the purpose to demonstrate the usefulness of EBMA, provides additional evidence for US presidential election forecasting. For this task, the error of the EBMA forecasts was 31% higher than the corresponding error the simple average. Simple averages produce accurate forecasts, are easy to describe, easy to understand, and easy to use. Researchers who develop new methods for combining forecasts need to compare the accuracy of their method to this widely established benchmark method. Forecasting practitioners should favor simple averages over more complex methods unless there is strong evidence in support of differential weights.