The role of the information set for forecasting—with applications to risk management

Predictions are issued on the basis of certain information. If the forecasting mechanisms are correctly specified, a larger amount of available information should lead to better forecasts. For point forecasts, we show how the effect of increasing the information set can be quantified by using strictly consistent scoring functions, where it results in smaller average scores. Further, we show that the classical Diebold-Mariano test, based on strictly consistent scoring functions and asymptotically ideal forecasts, is a consistent test for the effect of an increase in a sequence of information sets on $h$-step point forecasts. For the value at risk (VaR), we show that the average score, which corresponds to the average quantile risk, directly relates to the expected shortfall. Thus, increasing the information set will result in VaR forecasts which lead on average to smaller expected shortfalls. We illustrate our results in simulations and applications to stock returns for unconditional versus conditional risk management as well as univariate modeling of portfolio returns versus multivariate modeling of individual risk factors. The role of the information set for evaluating probabilistic forecasts by using strictly proper scoring rules is also discussed.

[1]  Andrew Samuels,et al.  Comparison and evaluation , 1986 .

[2]  Francis X. Diebold,et al.  Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold–Mariano Tests , 2012 .

[3]  A. Raftery,et al.  Probabilistic forecasts, calibration and sharpness , 2007 .

[4]  R. Durrett.  Probability: Theory and Examples , 1993 .

[5]  Stephen E. Fienberg,et al.  The Comparison and Evaluation of Forecasters. , 1983 .

[6]  Andrew J. Patton,et al.  Forecast Rationality Tests Based on Multi-Horizon Bounds , 2011 .

[7]  Jeremy Berkowitz,et al.  Evaluating Value-at-Risk Models with Desk-Level Data , 2007, Manag. Sci..

[8]  A. Raftery,et al.  Strictly Proper Scoring Rules, Prediction, and Estimation , 2007 .

[9]  C. Heinrich,et al.  The mode functional is not elicitable , 2014 .

[10]  J. Brocker.  Reliability, Sufficiency, and the Decomposition of Proper Scores , 2008, 0806.0813.

[11]  F. Diebold,et al.  Comparing Predictive Accuracy , 1994, Business Cycles.

[12]  Tilmann Gneiting,et al.  Evaluating Predictive Performance , 2014 .

[13]  Yong Bao,et al.  Evaluating Predictive Performance of Value-at-Risk Models in Emerging Markets: A Reality Check , 2006 .

[14]  Achim Klenke,et al.  Probability theory - a comprehensive course , 2008, Universitext.

[15]  Peter Christoffersen,et al.  Value–at–Risk Models , 2009 .

[16]  Peter F. CHRISTOFFERSENti.  EVALUATING INTERVAL FORECASTS , 2016 .

[17]  Halbert White,et al.  Tests of Conditional Predictive Ability , 2003 .

[18]  James Mitchell,et al.  Evaluating density forecasts: forecast combinations, model mixtures, calibration and sharpness , 2011 .

[19]  T. Gneiting,et al.  Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules , 2011 .

[20]  R. Rockafellar,et al.  Optimization of conditional value-at risk , 2000 .

[21]  P. Burridge,et al.  A Very Simple, Positive Semi-Definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix , 1991 .

[22]  Phhilippe Jorion.  Value at Risk: The New Benchmark for Managing Financial Risk , 2000 .

[23]  T. Gneiting.  Making and Evaluating Point Forecasts , 2009, 0912.0902.

[24]  D. Tasche,et al.  On the coherence of expected shortfall , 2001, cond-mat/0104295.

[25]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[26]  Alexander Tsyplakov,et al.  Evaluating Density Forecasts: A Comment , 2011 .

[27]  Christine M. Anderson-Cook,et al.  Book review: quantitative risk management: concepts, techniques and tools, revised edition, by A.F. McNeil, R. Frey and P. Embrechts. Princeton University Press, 2015, ISBN 978-0-691-16627-8, xix + 700 pp. , 2017, Extremes.

[28]  J. Carlos Escanciano,et al.  Robust Backtesting Tests for Value-at-Risk Models , 2008 .