Forecast Encompassing Tests and Probability Forecasts

We consider tests of forecast encompassing for probability forecasts, for both quadratic and logarithmic scoring rules. We propose test statistics for the null of forecast encompassing, present the limiting distributions of the test statistics, and investigate the impact of estimating the forecasting models’ parameters on these distributions. The small-sample performance of the various statistics is investigated, both in terms of small numbers of forecasts and model estimation sample sizes. Two empirical applications show the usefulness of the tests for the evaluation of recession probability forecasts from logit models with different leading indicators as explanatory variables, and for evaluating survey-based probability forecasts. Probability forecasts ; encompassing tests ; recession probabilities

[1]  M. E. Johnson,et al.  A Family of Distributions for Modelling Non‐Elliptically Symmetric Multivariate Data , 1981 .

[2]  J. Stock,et al.  A Comparison of Linear and Nonlinear Univariate Models for Forecasting Macroeconomic Time Series , 1998 .

[3]  Ivana Komunjer,et al.  Evaluation and Combination of Conditional Quantile Forecasts , 2002 .

[4]  H. White.  Asymptotic theory for econometricians , 1985 .

[5]  James D. Hamilton.  Oil and the Macroeconomy since World War II , 1983, Journal of Political Economy.

[6]  Farshid Vahid,et al.  PREDICTING THE PROBABILITY OF A RECESSION WITH NONLINEAR AUTOREGRESSIVE LEADING-INDICATOR MODELS , 2001, Macroeconomic Dynamics.

[7]  James D. Hamilton,et al.  A Reexamination of the Predictability of Economic Activity Using the Yield Spread , 2000 .

[8]  Jonathan H. Wright,et al.  Federal Reserve Board , 2000 .

[9]  C. Granger,et al.  Economic and Statistical Measures of Forecast Accuracy , 1999 .

[10]  Predicting A Recession: Evidence From The Yield Curve In The Presence Of Structural Breaks , 2002 .

[11]  Christian Genest,et al.  Combining Probability Distributions: A Critique and an Annotated Bibliography , 1986 .

[12]  C. Granger,et al.  Experience with Forecasting Univariate Time Series and the Combination of Forecasts , 1974 .

[13]  Peter F. Christoffersen.  Evaluating Interval Forecasts , 1998 .

[14]  M. Hooker,et al.  What happened to the oil price-macroeconomy relationship? , 1996 .

[15]  Kenneth F. Wallis,et al.  Chi-Squared Tests of Interval and Density Forecasts, and the Bank of England's Fan Charts , 2001, SSRN Electronic Journal.

[16]  C. Granger,et al.  Interval forecasting. An analysis based upon ARCH-quantile estimators , 1989 .

[17]  Peter E. Kennedy,et al.  Combining Qualitative Forecasts Using Logit , 1998 .

[18]  Anthony S. Tay,et al.  Evaluating Density Forecasts with Applications to Financial Risk Management , 1998 .

[19]  Ray C. Fair,et al.  Comparing Information in Forecasts from Econometric Models , 1990 .

[20]  G. Brier.  VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[21]  Michael P. Clements,et al.  Can oil shocks explain asymmetries in the US Business Cycle? , 2002 .

[22]  Paul Newbold,et al.  Some comments on the evaluation of economic forecasts , 1973 .

[23]  Chris Birchenhall,et al.  Predicting U.S. Business-Cycle Regimes , 1999 .

[24]  D. Hendry,et al.  Econometric Evaluation of Linear Macro-Economic Models , 1986 .

[25]  Chris Chatfield,et al.  Calculating Interval Forecasts , 1993 .

[26]  Francis X. Diebold,et al.  Scoring the Leading Indicators , 2020, Business Cycles.

[27]  Simon M. Potter,et al.  Predicting a Recession: Evidence from the Yield Curve in the Presence of Structural Breaks , 2001 .

[28]  Robert Fildes,et al.  Forecasting competitions - their role in improving forecasting practice and research , 2007 .

[29]  Jeremy Piger,et al.  The Use and Abuse of Real-Time Data in Economic Forecasting , 2003, Review of Economics and Statistics.

[30]  Michael P. Clements,et al.  On the limitations of comparing mean square forecast errors , 1993 .

[31]  F. Diebold,et al.  Comparing Predictive Accuracy , 1994, Business Cycles.

[32]  Michael P. Clements,et al.  Combining Predictors and Combining Information in Modelling: Forecasting US Recession Probabilities and Output Growth , 2006 .

[33]  James D. Hamilton.  This is what happened to the oil price-macroeconomy relationship , 1996 .

[34]  Kiseok Lee,et al.  Oil Shocks and the Macroeconomy: The Role of Price Variability* , 1995 .

[35]  Knut Anton Mork,et al.  Oil and the Macroeconomy When Prices Go Up and Down: An Extension of Hamilton's Results , 1989, Journal of Political Economy.

[36]  Kenneth D West,et al.  Tests for Forecasts Encompassing When Forecasts Depend on Estimated Regression Parameters , 2001 .

[37]  Todd E. Clark,et al.  Tests of Equal Forecast Accuracy and Encompassing for Nested Models , 1999 .

[38]  Kaplan,et al.  ‘Combining Probability Distributions from Experts in Risk Analysis’ , 2000, Risk analysis : an official publication of the Society for Risk Analysis.

[39]  C. Nelson.  The Prediction Performance of the FRB-MIT-PENN Model of the U.S. Economy , 1972 .

[40]  Paul Newbold,et al.  Tests for multiple forecast encompassing , 2000 .

[41]  K. West,et al.  Asymptotic Inference about Predictive Ability , 1996 .

[42]  Jonathan H. Wright,et al.  Exchange Rate Forecasting: The Errors We've Really Made , 2001 .

[43]  R. Clemen.  Combining forecasts: A review and annotated bibliography , 1989 .

[44]  Gerrit De Geest,et al.  The international library of critical writings in economics , 2004 .

[45]  Ken West,et al.  Chapter 3 Forecast Evaluation , 2006 .

[46]  Andrew J. Oswald,et al.  Unemployment Equilibria and Input Prices: Theory and Evidence from the United States , 1998, Review of Economics and Statistics.

[47]  A. H. Murphy,et al.  Scoring rules and the evaluation of probabilities , 1996 .

[48]  Kenneth F. Wallis,et al.  Density Forecasting: A Survey , 2000 .

[49]  James D. Hamilton.  What is an Oil Shock? , 2000 .

[50]  P. Newbold,et al.  Tests for Forecast Encompassing , 1998 .

[51]  Frederic S. Mishkin,et al.  Predicting U.S. Recessions: Financial Variables as Leading Indicators , 1995, Review of Economics and Statistics.

[52]  Dean Croushore,et al.  A real-time data set for macroeconomists , 2001 .

[53]  F. Diebold,et al.  Forecast Evaluation and Combination , 1996 .

[54]  Neil R. Ericsson.  On the limitations of comparing mean square forecast errors: Clarifications and extensions , 1993 .