Elicitability and backtesting: Perspectives for banking regulation

Conditional forecasts of risk measures play an important role in internal risk management of financial institutions as well as in regulatory capital calculations. In order to assess forecasting performance of a risk measurement procedure, risk measure forecasts are compared to the realized financial losses over a period of time and a statistical test of correctness of the procedure is conducted. This process is known as backtesting. Such traditional backtests are concerned with assessing some optimality property of a set of risk measure estimates. However, they are not suited to compare different risk estimation procedures. We investigate the proposal of comparative backtests, which are better suited for method comparisons on the basis of forecasting accuracy, but necessitate an elicitable risk measure. We argue that supplementing traditional backtests with comparative backtests will enhance the existing trading book regulatory framework for banks by providing the correct incentive for accuracy of risk measure forecasts. In addition, the comparative backtesting framework could be used by banks internally as well as by researchers to guide selection of forecasting methods. The discussion focuses on three risk measures, Value-at-Risk, expected shortfall and expectiles, and is supported by a simulation study and data analysis.

[1]  T. Gneiting.  Making and Evaluating Point Forecasts , 2009, 0912.0902.

[2]  Furno Marilena,et al.  Quantile Regression , 2018, Wiley Series in Probability and Statistics.

[3]  Alexander Tsyplakov,et al.  Theoretical guidelines for a partially informed forecast examiner , 2014 .

[4]  Tilmann Gneiting,et al.  Expected Shortfall is jointly elicitable with Value at Risk - Implications for backtesting , 2015, 1507.00244.

[5]  J. Corcoran.  Modelling Extremal Events for Insurance and Finance , 2002 .

[6]  DuZaichao,et al.  Backtesting Expected Shortfall , 2017 .

[7]  Nick Costanzino,et al.  Backtesting General Spectral Risk Measures with Application to Expected Shortfall , 2015 .

[8]  M. Steel,et al.  On Bayesian Modelling of Fat Tails and Skewness , 1998 .

[9]  M. Rosenblatt.  Remarks on a Multivariate Transformation , 1952 .

[10]  Anthony S. Tay,et al.  Evaluating Density Forecasts with Applications to Financial Risk Management , 1998 .

[11]  Johanna F. Ziegel,et al.  COHERENCE AND ELICITABILITY , 2013, 1303.1690.

[12]  Fabio Bellini,et al.  Elicitable Risk Measures , 2014 .

[13]  P. Embrechts,et al.  Quantitative Risk Management: Concepts, Techniques, and Tools , 2005 .

[14]  Mark H. A. Davis,et al.  Verification of internal risk measure estimates , 2014, 1410.4382.

[15]  B. Hansen.  Autoregressive Conditional Density Estimation , 1994 .

[16]  Rama Cont,et al.  Robustness and sensitivity analysis of risk measurement procedures , 2010 .

[17]  Tilmann Gneiting,et al.  Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings , 2015, 1503.08195.

[18]  Johanna F. Ziegel,et al.  Cross-calibration of probabilistic forecasts , 2015 .

[19]  B. Rémillard,et al.  Relating quantiles and expectiles under weighted-symmetry , 1995 .

[20]  A. Müller,et al.  Generalized Quantiles as Risk Measures , 2013 .

[21]  Marc S. Paolella,et al.  Value-at-Risk Prediction: A Comparison of Alternative Strategies , 2005 .

[22]  G. Hommel.  Tests of the overall hypothesis for arbitrary dependence structures , 1983 .

[23]  S. Weber,et al.  DISTRIBUTION‐INVARIANT RISK MEASURES, INFORMATION, AND DYNAMIC CONSISTENCY , 2006 .

[24]  A. McNeil,et al.  Estimation of tail-related risk measures for heteroscedastic financial time series: an extreme value approach , 2000 .

[25]  Hajo Holzmann,et al.  The role of the information set for forecasting—with applications to risk management , 2014, 1404.7653.

[26]  Dongming Zhu,et al.  A Generalized Asymmetric Student-t Distribution with Application to Financial Econometrics , 2009 .

[27]  Alexander Shapiro,et al.  Conditional Value-at-Risk and Average Value-at-Risk: Estimation and Asymptotics , 2012, Oper. Res..

[28]  J. Wooldridge,et al.  Quasi-maximum likelihood estimation and inference in dynamic models with time-varying covariances , 1992 .

[29]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[30]  Peter Christoffersen,et al.  Elements of Financial Risk Management , 2003 .

[31]  F. Diebold,et al.  Comparing Predictive Accuracy , 1994, Business Cycles.

[32]  Johanna F. Ziegel,et al.  Higher order elicitability and Osband’s principle , 2015, 1503.08123.

[33]  Fabio Bellini,et al.  Risk management with expectiles , 2014 .

[34]  Yoav Shoham,et al.  Eliciting properties of probability distributions , 2008, EC '08.

[35]  Discussion of “Elicitability and backtesting: Perspectives for banking regulation” , 2017 .

[36]  William Thomson,et al.  Eliciting production possibilities from a well-informed manager , 1979 .

[37]  F. Diebold,et al.  Pitfalls and Opportunities in the Use of Extreme Value Theory in Risk Management , 1998 .

[38]  Valeria Bignozzi,et al.  Risk measures with the CxLS property , 2014, Finance Stochastics.

[39]  Steven Kou,et al.  On the Measurement of Economic Tail Risk , 2014, Oper. Res..

[40]  Ruodu Wang,et al.  Elicitable distortion risk measures: A concise proof , 2015 .

[41]  Peter F. CHRISTOFFERSENti.  EVALUATING INTERVAL FORECASTS , 2016 .

[42]  Michael Curran.  Backtesting Expected Shortfall , 2014 .

[43]  T. Gneiting,et al.  Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules , 2011 .

[44]  Kevin Sheppard,et al.  Evaluating Volatility and Correlation Forecasts , 2009 .

[45]  Halbert White,et al.  Tests of Conditional Predictive Ability , 2003 .

[46]  Siyu Zhang,et al.  Elicitation and Identification of Properties , 2014, COLT.

[47]  W. Newey,et al.  Asymmetric Least Squares Estimation and Testing , 1987 .

[48]  R. Nau.  Should Scoring Rules be Effective , 1985 .

[49]  Andrew J. Patton.  Volatility Forecast Comparison Using Imperfect Volatility Proxies , 2006 .

[50]  Alexander Schied,et al.  Convex measures of risk and trading constraints , 2002, Finance Stochastics.

[51]  Marco Saerens,et al.  Building cost functions minimizing to some summary statistics , 2000, IEEE Trans. Neural Networks Learn. Syst..

[52]  Kent Osband,et al.  Providing incentives for better cost forecasting , 1985 .

[53]  Ian A. Kash,et al.  Vector-Valued Property Elicitation , 2015, COLT.

[54]  M. Kratz,et al.  What is the Best Risk Measure in Practice? A Comparison of Standard Measures , 2013, 1312.1645.

[55]  Chung-Ming Kuan,et al.  Assessing Value at Risk With CARE, the Conditional Autoregressive Expectile Models , 2008 .