An evaluation of tests of distributional forecasts

One popular method for testing the validity of a model's forecasts is to use the probability integral transforms (pits) of the forecasts and to test for departures from the dual hypotheses of independence and uniformity, with departures from uniformity tested using the Kolmogorov-Smirnov (KS) statistic. This paper investigates the power of five statistics (including the KS statistic) to reject uniformity of the pits in the presence of misspecification in the mean, variance, skewness or kurtosis of the forecast errors. The KS statistic has the lowest power of the five statistics considered and is always dominated by the Anderson and Darling statistic. Copyright (C) 2003 John Wiley Sons, Ltd.

[1]  Emanuela Marrocu,et al.  The performance of non‐linear exchange rate models: a forecasting comparison , 2002 .

[2]  Michael P. Clements,et al.  Evaluating The Forecast of Densities of Linear and Non-Linear Models: Applications to Output Growth and Unemployment , 2000 .

[3]  Anthony S. Tay,et al.  Evaluating Density Forecasts of Inflation: The Survey of Professional Forecasters , 1997 .

[4]  G. S. Watson,et al.  Goodness-of-fit tests on a circle. II , 1961 .

[5]  Kenneth F. Wallis,et al.  Chi-Squared Tests of Interval and Density Forecasts, and the Bank of England's Fan Charts , 2001, SSRN Electronic Journal.

[6]  D. Darling,et al.  A Test of Goodness of Fit , 1954 .

[7]  Anthony S. Tay,et al.  Evaluating Density Forecasts with Applications to Financial Risk Management , 1998 .

[8]  Gordon Anderson,et al.  Simple tests of distributional form , 1994 .

[9]  A. P. Dawid,et al.  Present position and potential developments: some personal views , 1984 .

[10]  M. Rosenblatt.  Remarks on a Multivariate Transformation , 1952 .

[11]  F. Diebold,et al.  Forecast Evaluation and Combination , 1996 .

[12]  N. Kuiper.  Tests concerning random points on a circle , 1960 .

[13]  Pandu R. Tadikamalla,et al.  A Probability Distribution and its Uses in Fitting Data , 1979 .

[14]  M. Stephens.  EDF Statistics for Goodness of Fit and Some Comparisons , 1974 .

[15]  Jeremy Berkowitz.  Testing Density Forecasts, With Applications to Risk Management , 2001 .

[16]  Peter F. Christoffersen.  Evaluating Interval Forecasts , 1998 .

[17]  H. Neave.  Distribution-Free Tests , 1988 .

[18]  Anthony S. Tay,et al.  Multivariate Density Forecast Evaluation and Calibration In Financial Risk Management: High-Frequency Returns on Foreign Exchange , 1999, Review of Economics and Statistics.