Comparing Density Forecast Models

In this paper we discuss how to compare various (possibly misspecified) density forecast models using the Kullback-Leibler information criterion (KLIC) of a candidate density forecast model with respect to the true density. The KLIC differential between a pair of competing models is the (predictive) log-likelihood ratio (LR) between the two models. Even though the true density is unknown, using the LR statistic amounts to comparing models with the KLIC as a loss function and thus enables us to assess which density forecast model can approximate the true density more closely. We also discuss how this KLIC is related to the KLIC based on the probability integral transform (PIT) in the framework of Diebold et al. (1998). While they are asymptotically equivalent, the PIT-based KLIC is best suited for evaluating the adequacy of each density forecast model and the original KLIC is best suited for comparing competing models. In an empirical study with the S&P500 and NASDAQ daily return series, we find strong evidence for rejecting the normal-GARCH benchmark model, in favor of the models that can capture skewness in the conditional distribution and asymmetry and long memory in the conditional variance. Copyright © 2007 John Wiley & Sons, Ltd.

[1]  James Mitchell,et al.  Evaluating, Comparing and Combining Density Forecasts Using the Klic with an Application to the Bank of England and Niesr 'Fan' Charts of Inflation , 2005 .

[2]  P. Hansen.  A Test for Superior Predictive Ability , 2005 .

[3]  Luc Bauwens,et al.  A Comparison of Financial Duration Models Via Density Forecast , 2004 .

[4]  Norman R. Swanson,et al.  Predictive Density and Conditional Confidence Interval Accuracy Tests , 2004 .

[5]  Yong Bao,et al.  A Test for Density Forecast Comparison with Applications to Risk Management , 2004 .

[6]  Tae-Hwy Lee,et al.  DIAGNOSTIC CHECKING FOR THE ADEQUACY OF NONLINEAR TIME SERIES MODELS , 2003, Econometric Theory.

[7]  Evaluating credit risk models using loss density forecasts , 2003 .

[8]  C. Granger,et al.  Forecasting Volatility in Financial Markets: A Review , 2003 .

[9]  Paolo Vanini,et al.  Operational Risk: A Practitioner's View , 2002 .

[10]  Giorgio Valente,et al.  Comparing the accuracy of density forecasts from competing models , 2002 .

[11]  Raffaella Giacomini,et al.  Comparing Density Forecasts via Weighted Likelihood Ratio Tests , 2002 .

[12]  James Davidson,et al.  Moment and Memory Properties of Linear Conditional Heteroscedasticity Models, and a New Model , 2004 .

[13]  Jin Wang,et al.  Generating daily changes in market variables using a multivariate mixture of normal distributions , 2001, Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304).

[14]  Jeremy Berkowitz.  Testing Density Forecasts, With Applications to Risk Management , 2001 .

[15]  M. Rockinger,et al.  Gram–Charlier densities , 2001 .

[16]  Luca Benzoni,et al.  An Empirical Investigation of Continuous-Time Equity Return Models , 2001 .

[17]  Stephen Gray,et al.  Semiparametric ARCH models , 2001 .

[18]  H. White,et al.  A Reality Check for Data Snooping , 2000 .

[19]  I. Mauleón,et al.  Testing densities with financial data: an empirical comparison of the EdgeworthSargan density to the Students t , 2000 .

[20]  Yihong Xia.  Learning About Predictability: The Effects of Parameter Uncertainty on Dynamic Asset Allocation , 2000 .

[21]  Clive W. J. Granger,et al.  A Decision_Theoretic Approach to Forecast Evaluation , 2000 .

[22]  N. Barberis.  Investing for the Long Run When Returns are Predictable , 2000 .

[23]  C. Granger,et al.  Economic and Statistical Measures of Forecast Accuracy , 1999 .

[24]  Juan Romo,et al.  Effects of parameter estimation on prediction densities: a bootstrap approach , 1999 .

[25]  Michael P. Clements,et al.  Evaluating The Forecast of Densities of Linear and Non-Linear Models: Applications to Output Growth and Unemployment , 2000 .

[26]  W. N. Street,et al.  Financial Data and the Skewed Generalized T Distribution , 1998 .

[27]  Anthony S. Tay,et al.  Evaluating Density Forecasts with Applications to Financial Risk Management , 1998 .

[28]  G. González-Rivera,et al.  Smooth-Transition GARCH Models , 1998 .

[29]  Subu Venkataraman,et al.  Value at risk for a mixture of normal distributions: the use of quasi- Bayesian estimation techniques , 1997 .

[30]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[31]  M. Steel,et al.  On Bayesian Modelling of Fat Tails and Skewness , 1998 .

[32]  M. Rubinstein.,et al.  Recovering Probability Distributions from Option Prices , 1996 .

[33]  C. Granger,et al.  Modeling volatility persistence of speculative returns: A new approach , 1996 .

[34]  T. Bollerslev,et al.  MODELING AND PRICING LONG- MEMORY IN STOCK MARKET VOLATILITY , 1996 .

[35]  John Knight,et al.  Statistical modelling of asymmetric risk in asset returns , 1995 .

[36]  Svetlozar T. Rachev,et al.  Stable GARCH models for financial time series , 1995 .

[37]  E. Eberlein,et al.  Hyperbolic distributions in finance , 1995 .

[38]  James B. McDonald,et al.  A generalization of the beta distribution with applications , 1995 .

[39]  R. Stambaugh,et al.  On the Predictability of Stock Returns: An Asset-Allocation Perspective , 1995 .

[40]  K. West,et al.  Asymptotic Inference about Predictive Ability , 1996 .

[41]  Joseph P. Romano,et al.  The stationary bootstrap , 1994 .

[42]  J. Zakoian.  Threshold heteroskedastic models , 1994 .

[43]  Halbert White,et al.  Estimation, inference, and specification analysis , 1996 .

[44]  L. Glosten,et al.  On the Relation between the Expected Value and the Volatility of the Nominal Excess Return on Stocks , 1993 .

[45]  C. Granger,et al.  A long memory property of stock market returns and a new model , 1993 .

[46]  S. Rachev,et al.  Modeling asset returns with alternative stable distributions , 1993 .

[47]  Daniel B. Nelson.  CONDITIONAL HETEROSKEDASTICITY IN ASSET RETURNS: A NEW APPROACH , 1991 .

[48]  Q. Vuong.  Likelihood Ratio Tests for Model Selection and Non-Nested Hypotheses , 1989 .

[49]  James B. McDonald,et al.  Partially Adaptive Estimation of Regression Models via the Generalized T Distribution , 1988, Econometric Theory.

[50]  A. Gallant,et al.  Semi-nonparametric Maximum Likelihood Estimation , 1987 .

[51]  T. Bollerslev,et al.  Generalized autoregressive conditional heteroskedasticity , 1986 .

[52]  R. Engle.  Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation , 1982 .

[53]  H. White.  Maximum Likelihood Estimation of Misspecified Models , 1982 .

[54]  John Matatko,et al.  Estimation risk and optimal portfolio choice , 1980 .

[55]  Takamitsu Sawa,et al.  Information criteria for discriminating among alternative regression models / BEBR No. 455 , 1978 .