Combining the Forecasts in the ECB Survey of Professional Forecasters: Can Anything Beat the Simple Average?

In this paper, we explore the potential gains from alternative combinations of the surveyed forecasts in the ECB Survey of Professional Forecasters. Our analysis encompasses a variety of methods including statistical combinations based on principal components analysis and trimmed means, performance-based weighting, least squares estimates of optimal weights as well as Bayesian shrinkage. We provide a pseudo real-time out-of-sample performance evaluation of these alternative combinations and check the sensitivity of the results to possible data-snooping bias. The latter robustness check is also informed using a novel real time meta selection procedure which is not subject to the data-snooping critique. For GDP growth and the unemployment rate, only few of the forecast combination schemes are able to outperform the simple equal-weighted average forecast. Conversely, for the inflation rate there is stronger evidence that more refined combinations can lead to improvement over this benchmark. In particular, for this variable, the relative improvement appears significant even controlling for data snooping bias. JEL Classification: C22, C53

[1]  J. M. Bates,et al.  The Combination of Forecasts , 1969 .

[2]  F. Diebold,et al.  Structural change and the combination of forecasts , 1986 .

[3]  Stephen Taylor,et al.  Forecasting Economic Time Series , 1979 .

[4]  A. Timmermann.  Forecast Combinations , 2005 .

[5]  P. Hansen.  A Test for Superior Predictive Ability , 2005 .

[6]  J. Stock,et al.  Combination forecasts of output growth in a seven-country data set , 2004 .

[7]  R. L. Winkler,et al.  Combining Economic Forecasts , 1986 .

[8]  Todd E. Clark,et al.  Tests of equal predictive ability with Real-Time data. Discussion paper , 2007 .

[9]  Dean Croushore,et al.  A Real-Time Data Set for Macroeconomists: Does the Data Vintage Matter? , 1999 .

[10]  Min Wu,et al.  Technical Trading-Rule Profitability, Data Snooping, and Reality Check: Evidence from the Foreign Exchange Market , 2005 .

[11]  Allan Timmermann,et al.  Forecast Combination With Entry and Exit of Experts , 2006 .

[12]  C. Granger,et al.  Improved methods of combining forecasts , 1984 .

[13]  Francis X. Diebold,et al.  The use of prior information in forecast combination , 1990 .

[14]  Robert L. Winkler,et al.  The accuracy of extrapolation (time series) methods: Results of a forecasting competition , 1982 .

[15]  Allan Timmermann,et al.  Persistence in forecasting performance and conditional combination strategies , 2006 .

[16]  Dean Croushore,et al.  A Real-Time Data Set for Macroeconomists: Does the Data Vintage Matter? , 1999, Review of Economics and Statistics.

[17]  D. Giannone,et al.  An Area-Wide Real-Time Database for the Euro Area , 2010, Review of Economics and Statistics.

[18]  M. Neerman-Arbez,et al.  To aggregate or not to aggregate… , 2007, Journal of thrombosis and haemostasis : JTH.

[19]  Michael P. Clements,et al.  Pooling of Forecasts , 2004 .

[20]  K. Wallis,et al.  A Simple Explanation of the Forecast Combination Puzzle , 2009 .

[21]  C. Granger,et al.  Experience with Forecasting Univariate Time Series and the Combination of Forecasts , 1974 .

[22]  Luca Onorante,et al.  Short-Term Inflation Projections: A Bayesian Vector Autoregressive Approach , 2010 .

[23]  R. L. Winkler,et al.  Averages of Forecasts: Some Empirical Results , 1983 .

[24]  J. Stock,et al.  Macroeconomic Forecasting Using Diffusion Indexes , 2002 .

[25]  F. Diebold,et al.  Comparing Predictive Accuracy , 1994, Business Cycles.

[26]  K. West,et al.  Asymptotic Inference about Predictive Ability , 1996 .

[27]  D. Croushore,et al.  Philadelphia Fed forecasting surveys: their value for research , 2010 .

[28]  R. Clemen.  Combining forecasts: A review and annotated bibliography , 1989 .

[29]  Roberto S. Mariano,et al.  Testing Forecast Accuracy , 2007 .

[30]  Andrew J. Patton,et al.  New Tests of Forecast Optimality Across Multiple Horizons , 2010 .

[31]  Michael P. Clements,et al.  Forecast Combination and Encompassing , 2009 .

[32]  H. White,et al.  Data‐Snooping, Technical Trading Rule Performance, and the Bootstrap , 1999 .

[33]  H. White,et al.  A Reality Check for Data Snooping , 2000 .