Inference about predictive ability

In this paper we provide a brief review of how out-of-sample methods can be used to construct tests that evaluate a time-series model's ability to predict. We focus on the role that parameter estimation plays in constructing asymptotically valid tests of predictive ability. We illustrate why forecasts and forecast errors that depend upon estimated parameters may have statistical properties that differ from those of their population counterparts. We explain how to conduct asymptotic inference, taking due account of dependence on estimated parameters.

[1]  Ray C. Fair,et al.  The Informational Context of Ex Ante Forecasts , 1989 .

[2]  Kenneth Rogoff,et al.  Was It Real? The Exchange Rate‐Interest Differential Relation over the Modern Floating‐Rate Period , 1988 .

[3]  A. Goldberger,et al.  Impact Multipliers and Dynamic Properties of the Klein-Goldberger Model. , 1961 .

[4]  Neil R. Ericsson,et al.  Encompassing the forecasts of U.S. trade balance models , 1993 .

[5]  J. M. Bates,et al.  The Combination of Forecasts , 1969 .

[6]  Norman R. Swanson,et al.  Money and output viewed through a rolling window , 1998 .

[7]  Y. P. Chung,et al.  The predictability of stock returns – a nonparametric approach , 1996 .

[8]  Emi Nakamura,et al.  Inflation forecasting using a neural network , 2005 .

[9]  C. Granger,et al.  Improved methods of combining forecasts , 1984 .

[10]  Menzie David Chinn,et al.  Banking on currency forecasts: How predictable is change in money? , 1995 .

[11]  A. P. Dawid,et al.  Present position and potential developments: some personal views , 1984 .

[12]  Ted Jaditz,et al.  Out-of-Sample Forecast Performance as a Test for Nonlinearity in Time Series , 1998 .

[13]  Francis X. Diebold,et al.  Forecast combination and encompassing: Reconciling two divergent literatures , 1989 .

[14]  S. Blomberg,et al.  Politics and exchange rate forecasts , 1997 .

[15]  Simon van Norden,et al.  Terms of trade and real exchange rates: the Canadian evidence , 1995 .

[16]  M. Hashem Pesaran,et al.  A Simple Nonparametric Test of Predictive Performance , 1992 .

[17]  A. Lo,et al.  MAXIMIZING PREDICTABILITY IN THE STOCK AND BOND MARKETS , 1995, Macroeconomic Dynamics.

[18]  A. Timmermann,et al.  Predictability of Stock Returns: Robustness and Economic Significance , 1995 .

[19]  D. Orden,et al.  Granger Causality From The Exchange Rate To Agricultural Prices And Export Sales , 1990 .

[20]  A. Tegene,et al.  Evaluating forecasting models of farmland prices , 1994 .

[21]  L. Glosten,et al.  Economic Significance of Predictable Variations in Stock Index Returns , 1989 .

[22]  Stephen K McNees.  The "Rationality" of Economic Forecasts , 1978 .

[23]  Michio Hatanaka.  A Simple Suggestion to Improve the Mincer-Zarnowitz Criterion for the Evaluation for Forecasts , 1974 .

[24]  Norman R. Swanson,et al.  A Model Selection Approach to Real-Time Macroeconomic Forecasting Using Linear Models and Artificial Neural Networks , 1997, Review of Economics and Statistics.

[25]  Adrian Pagan,et al.  Alternative Models for Conditional Stock Volatility , 1989 .

[26]  T. Day,et al.  Stock market volatility and the information content of stock index options , 1992 .

[27]  James G. MacKinnon,et al.  Model Specification Tests Based on Artificial Linear Regressions , 1984 .

[28]  Todd E. Clark.  Finite‐sample properties of tests for equal forecast accuracy , 1999 .

[29]  K. West,et al.  Asymptotic Inference about Predictive Ability , 1996 .

[30]  R. Shiller,et al.  The Informational Content of Ex Ante Forecasts , 1988 .

[31]  Jeffrey S. Racine,et al.  Semiparametric ARX neural-network models with an application to forecasting inflation , 2001, IEEE Trans. Neural Networks.

[32]  Norman R. Swanson,et al.  Predictive ability with cointegrated variables , 2001 .

[33]  Norman R. Swanson,et al.  Forecasting economic time series using flexible versus fixed specification and linear versus nonlinear econometric models , 1997 .

[34]  Michael W. McCracken,et al.  New Mse Tests For Evaluating Forecasting Performance: Empirics And Bootstrap , 2001 .

[35]  Stephen D. Oliner,et al.  New and Old Models of Business Investment: A Comparison of Forecasting Performance , 1995 .

[36]  Clive W. J. Granger,et al.  A Decision_Theoretic Approach to Forecast Evaluation , 2000 .

[37]  R. Cumby,et al.  Testing for market timing ability: A framework for forecast evaluation , 1987 .

[38]  H. White,et al.  A Reality Check for Data Snooping , 2000 .

[39]  Gordon Leitch,et al.  Economic Forecast Evaluation: Profits versus the Conventional Error Measures , 1991 .

[40]  Sydney C. Ludvigson,et al.  Does Consumer Confidence Forecast Household Expenditure? A Sentiment Index Horse Race , 1998 .

[41]  A. Goldberger.  A course in econometrics , 1991 .

[42]  Richard Ashley.  Inflation and the Distribution of Price Changes across Markets: A Causal Analysis , 1981 .

[43]  W. A. Morgan.  TEST FOR THE SIGNIFICANCE OF THE DIFFERENCE BETWEEN THE TWO VARIANCES IN A SAMPLE FROM A NORMAL BIVARIATE POPULATION , 1939 .

[44]  Norman R. Swanson,et al.  A Model-Selection Approach to Assessing the Information in the Term Structure Using Linear Models and Artificial Neural Networks , 1995 .

[45]  J. Stock,et al.  Interpreting Evidence on Money-Income Causality , 1987 .

[46]  Glenn D. Rudebusch,et al.  Forecasting Output with the Composite Leading Index: A Real-Time Analysis , 1991 .

[47]  Joel H. Steckel,et al.  Cross-Validating Regression Models in Marketing Research , 1993 .

[48]  Dennis L. Hoffman,et al.  PRACTITIONERS CORNER: Post‐Sample Prediction Tests for Generalized Method of Moments Estimators , 2009 .

[49]  M. Hashem Pesaran,et al.  A generalization of the non-parametric Henriksson-Merton test of market timing , 1994 .

[50]  Evaluating density forecasts from models of stock market returns , 2005 .

[51]  C. Engel.  Can the Markov Switching Model Forecast Exchange Rates? , 1992 .

[52]  E. Ronchetti,et al.  Robust Tests of Predictive Accuracy , 2002 .

[53]  A. H. Murphy,et al.  Probability, Statistics, And Decision Making In The Atmospheric Sciences , 1985 .

[54]  J. Stock,et al.  A Procedure for Predicting Recessions with Leading Indicators: Econometric Issues and Recent Experience , 1992 .

[55]  Ronald H. Randles,et al.  On the Asymptotic Normality of Statistics with Estimated Parameters , 1982 .

[56]  Ray C. Fair,et al.  Estimating the Expected Predictive Accuracy of Econometric Models , 1980 .

[57]  M. Keane,et al.  Are economic forecasts rational , 1989 .

[58]  Michael P. Clements,et al.  Guest Editors' Introduction: Information in Economic Forecasting , 2005 .

[59]  Chung-Ming Kuan,et al.  Forecasting exchange rates using feedforward and recurrent neural networks , 1992 .

[60]  James Durbin,et al.  Testing for Serial Correlation in Least-Squares Regression When Some of the Regressors are Lagged Dependent Variables , 1970 .

[61]  Jon A. Brandt,et al.  Price forecasting and evaluation: An application in agriculture , 1983 .

[62]  D. Hendry,et al.  Econometric Evaluation of Linear Macro-Economic Models , 1986 .

[63]  Neil R. Ericsson.  Parameter constancy, mean square forecast errors, and measuring forecast performance: an exposition, extensions, and illustration , 1991 .

[64]  F. Diebold,et al.  Comparing Predictive Accuracy , 1994, Business Cycles.

[65]  Norman R. Swanson,et al.  An introduction to stochastic unit-root processes , 1997 .

[66]  Richard Schmalensee,et al.  Advertising and aggregate consumption: an analysis of causality , 1980 .

[67]  Richard Schmalensee,et al.  Advertising and aggregate consumption: an analysis of causality , 1980 .

[68]  Michael W. McCracken.  Asymptotics for Out of Sample Tests of Causality , 1998 .

[69]  K. West,et al.  The Predictive Ability of Several Models of Exchange Rate Volatility , 1994 .

[70]  Michael P. Clements,et al.  Evaluating multivariate forecast densities: a comparison of two approaches , 2002 .

[71]  G. Brier.  VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[72]  P. Newbold,et al.  Tests for Forecast Encompassing , 1998 .

[73]  Timothy A. Park,et al.  Forecast Evaluation for Multivariate Time-Series Models: The U.S. Cattle Market , 1990 .

[74]  Paul Newbold,et al.  Forecast evaluation tests in the presence of ARCH , 1999 .

[75]  N. Mark,et al.  Exchange Rates and Fundamentals: Evidence on Long-Horizon Predictability , 1995 .

[76]  Peter E. Rossi,et al.  Posterior, predictive, and utility-based approaches to testing the arbitrage pricing theory , 1990 .

[77]  R. Fair.  Evaluating the Information Content and Money Making Ability of Forecasts from Exchange Rate Equations , 1997 .

[78]  Jose A. Lopez.  Evaluating the Predictive Accuracy of Volatility Models , 2001 .

[79]  Ray C. Fair,et al.  Comparing Information in Forecasts from Econometric Models , 1990 .

[80]  A. Lo,et al.  Data-Snooping Biases in Tests of Financial Asset Pricing Models , 1989 .

[81]  Pablo M. Pincheira.  Combining tests of predictive ability: theory and evidence for chilean and canadian exchange rates , 2008 .

[82]  Lutz Kilian,et al.  Exchange Rates and Monetary Fundamentals: What Do We Learn from Long-Horizon Regressions? , 1999 .

[83]  Richard Ashley.  A new technique for postsample model selection and validation , 1998 .

[84]  Adrian Pagan,et al.  Diagnostic tests as residual analysis , 1983 .

[85]  ON COMPARING MACROECONOMIC MODELS USING FORECAST ENCOMPASSING TESTS , 2009 .

[86]  Paul Newbold,et al.  Testing the equality of prediction mean squared errors , 1997 .

[87]  Comments on the evaluation of policy models , 1991 .

[88]  Michael W. McCracken.  Robust out-of-sample inference , 2000 .

[89]  Allen N. Berger,et al.  The Information Efficiency of Econometric Model Forecasts , 1985 .

[90]  Michael P. Clements,et al.  Forecasting economic and financial time-series with non-linear models , 2004 .

[91]  Employment, unemployment and the minimum wage: a causality model , 1988 .

[92]  Comments on the evaluation of policy models , 2001 .

[93]  Kenneth D West,et al.  Tests for Forecasts Encompassing When Forecasts Depend on Estimated Regression Parameters , 2001 .

[94]  K. West,et al.  A Utility Based Comparison of Some Models of Exchange Rate Volatility , 1992 .

[95]  C. Granger,et al.  Forecasting Economic Time Series. , 1988 .

[96]  Todd E. Clark,et al.  Tests of Equal Forecast Accuracy and Encompassing for Nested Models , 1999 .