Robust out-of-sample inference

This paper presents analytical, empirical and simulation results concerning inference about the moments of nondifferentiable functions of out-of-sample forecasts and forecast errors. Special attention is given to the measurement of a model's predictive ability using the test of equal mean absolute error. Tests for equal mean absolute error and mean square error are used to evaluate predictions of excess returns to the S & P 500 composite. Simulations indicate that appropriately constructed tests for equal mean absolute error can provide more accurately sized and more powerful tests than inappropriately constructed tests for equal mean absolute error and mean square error.

[1]  D. Pollard,et al.  Cube Root Asymptotics , 1990 .

[2]  Norman R. Swanson,et al.  A Model Selection Approach to Real-Time Macroeconomic Forecasting Using Linear Models and Artificial Neural Networks , 1997, Review of Economics and Statistics.

[3]  A. Timmermann,et al.  Predictability of Stock Returns: Robustness and Economic Significance , 1995 .

[4]  Clive W. J. Granger,et al.  Prediction with a generalized cost of error function , 1969 .

[5]  Paul Newbold,et al.  Forecast evaluation tests in the presence of ARCH , 1999 .

[6]  W. Newey,et al.  A Simple, Positive Semi-Definite, Heteroskedasticity and Autocorrelationconsistent Covariance Matrix , 1986 .

[7]  Norman R. Swanson,et al.  Predictive ability with cointegrated variables , 2001 .

[8]  M. Hashem Pesaran,et al.  A Simple Nonparametric Test of Predictive Performance , 1992 .

[9]  H. White.  Asymptotic theory for econometricians , 1985 .

[10]  J. Davidson.  Stochastic Limit Theory , 1994 .

[11]  Richard Schmalensee,et al.  Advertising and aggregate consumption: an analysis of causality , 1980 .

[12]  W. Newey,et al.  A Simple, Positive Semi-Definite, Heteroskedasticity and Autocorrelationconsistent Covariance Matrix , 1986 .

[13]  E. Fama,et al.  Dividend yields and expected stock returns , 1988 .

[14]  Norman R. Swanson,et al.  A Model-Selection Approach to Assessing the Information in the Term Structure Using Linear Models and Artificial Neural Networks , 1995 .

[15]  Chung-Ming Kuan,et al.  Forecasting exchange rates using feedforward and recurrent neural networks , 1992 .

[16]  Ray C. Fair,et al.  Comparing Information in Forecasts from Econometric Models , 1990 .

[17]  Associates Ibbotson,et al.  Stocks, bonds, bills, and inflation yearbook , 1993 .

[18]  H. White,et al.  A Reality Check for Data Snooping , 2000 .

[19]  M. Richardson.  Predictability of stock returns : Statistical theory and evidence , 1989 .

[20]  Donald B. Keim,et al.  Predicting returns in the stock and bond markets , 1986 .

[21]  S. Irwin,et al.  Economic evaluation of commodity price forecasting models , 1993 .

[22]  R. Shiller.  Stock Prices and Social Dynamics , 1984 .

[23]  V. Akgiray.  Conditional Heteroscedasticity in Time Series of Stock Returns: Evidence and Forecasts , 1989 .

[24]  Herman Stekler,et al.  Macroeconomic forecast evaluation techniques , 1991 .

[25]  F. Diebold,et al.  Comparing Predictive Accuracy , 1994, Business Cycles.

[26]  P. Hall,et al.  Martingale Limit Theory and its Application. , 1984 .

[27]  Central Limit Theorems For Dependent, Heterogeneous Processes With Trending Moments , 1989 .

[28]  R. Shiller,et al.  Stock Prices, Earnings and Expected Dividends , 1988 .

[29]  C. Engel.  Can the Markov Switching Model Forecast Exchange Rates? , 1992 .

[30]  Bruce E. Hansen,et al.  Consistent Covariance Matrix Estimation for Dependent Heterogeneous Processes , 1992 .

[31]  Kenneth Rogoff,et al.  Empirical exchange rate models of the seventies , 1983 .

[32]  G. Tassinari.  Advertising and aggregate consumption in Italy: 1981-1993 , 1995 .

[33]  Adrian Pagan,et al.  Alternative Models for Conditional Stock Volatility , 1989 .

[34]  Kenneth S. Rogoff,et al.  Exchange rate models of the seventies. Do they fit out of sample , 1983 .

[35]  Jeffrey S. Racine,et al.  Semiparametric ARX neural-network models with an application to forecasting inflation , 2001, IEEE Trans. Neural Networks.

[36]  E. Fama,et al.  Efficient Capital Markets : II , 2007 .

[37]  K. West,et al.  Asymptotic Inference about Predictive Ability , 1996 .

[38]  Andrew A. Weiss,et al.  Estimating Time Series Models Using the Relevant Cost Function , 1996 .

[39]  Ronald H. Randles,et al.  On the Asymptotic Normality of Statistics with Estimated Parameters , 1982 .