Robust Forecast Evaluation of Expected Shortfall

Motivated by the Basel III regulations, recent studies have considered joint forecasts of Value-at-Risk and Expected Shortfall. A large family of scoring functions can be used to evaluate forecast performance in this context. However, little intuitive or empirical guidance is currently available, which renders the choice of scoring function awkward in practice. We therefore develop graphical checks of whether one forecast method dominates another under a relevant class of scoring functions, and propose an associated hypothesis test. We illustrate these tools with simulation examples and an empirical analysis of S&P 500 and DAX returns.

[1]  T. Gneiting.  Quantiles as optimal point forecasts , 2011 .

[2]  Halbert White,et al.  Tests of Conditional Predictive Ability , 2003 .

[3]  W. Ehm,et al.  Forecast dominance testing via sign randomization , 2017, 1707.03035.

[4]  N. Shephard,et al.  Multivariate stochastic variance models , 1994 .

[5]  Neil Shephard,et al.  Designing Realised Kernels to Measure the Ex-Post Variation of Equity Prices in the Presence of Noise , 2008 .

[6]  T. Gneiting.  Making and Evaluating Point Forecasts , 2009, 0912.0902.

[7]  Tilmann Gneiting,et al.  Expected Shortfall is jointly elicitable with Value at Risk - Implications for backtesting , 2015, 1507.00244.

[8]  T. Bollerslev,et al.  Generalized autoregressive conditional heteroskedasticity , 1986 .

[9]  Nicolai Meinshausen,et al.  Asymptotic optimality of the Westfall--Young permutation procedure for multiple testing under dependence , 2011, 1106.2068.

[10]  Timo Dimitriadis,et al.  A joint quantile and expected shortfall regression framework , 2017, Electronic Journal of Statistics.

[11]  P. Hansen.  A Test for Superior Predictive Ability , 2005 .

[12]  Andrew J. Patton.  Volatility Forecast Comparison Using Imperfect Volatility Proxies , 2006 .

[13]  Hajo Holzmann,et al.  The role of the information set for forecasting—with applications to risk management , 2014, 1404.7653.

[14]  H. White,et al.  A Reality Check for Data Snooping , 2000 .

[15]  Johanna F. Ziegel,et al.  Elicitability and backtesting: Perspectives for banking regulation , 2016, 1608.05498.

[16]  Andrew J. Patton.  Comparing Possibly Misspecified Forecasts , 2020, Journal of Business & Economic Statistics.

[17]  N. Shephard,et al.  Realized Kernels in Practice: Trades and Quotes , 2009 .

[18]  Helmut Ltkepohl,et al.  New Introduction to Multiple Time Series Analysis , 2007 .

[19]  S. S. Young,et al.  Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment , 1993 .

[20]  Neil Shephard,et al.  Realising the future: forecasting with high frequency based volatility (HEAVY) models , 2010 .

[21]  Dennis D. Cox,et al.  Pointwise testing with functional data using the Westfall–Young randomization method , 2008 .

[22]  Joseph P. Romano,et al.  The stationary bootstrap , 1994 .

[23]  Tilmann Gneiting,et al.  Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings , 2015, 1503.08195.

[24]  Dobrislav Dobrev,et al.  Accurate Evaluation of Expected Shortfall for Linear Portfolios with Elliptically Distributed Risk Factors , 2017 .

[25]  F. Diebold,et al.  Comparing Predictive Accuracy , 1994, Business Cycles.

[26]  Rui Chen,et al.  Dynamic Semiparametric Models for Expected Shortfall (and Value-At-Risk) , 2017, Journal of Econometrics.

[27]  Freddy Delbaen,et al.  Monetary utility functions , 2012 .

[28]  Johanna F. Ziegel,et al.  Higher order elicitability and Osband’s principle , 2015, 1503.08123.

[29]  L. J. Savage.  Elicitation of Personal Probabilities and Expectations , 1971 .

[30]  A. H. Murphy,et al.  The Value of Climatological, Categorical and Probabilistic Forecasts in the Cost-Loss Ratio Situation , 1977 .

[31]  T. Yen,et al.  Testing Forecast Accuracy of Expectiles and Quantiles with the Extremal Consistent Loss Functions , 2017, International Journal of Forecasting.