Evaluating quantile-bounded and expectile-bounded interval forecasts

Abstract In many different contexts, decision-making is improved by the availability of probabilistic predictions. The accuracy of probabilistic forecasting methods can be compared using scoring functions and insight provided by calibration tests. These tests evaluate the consistency of predictions with the observations. Our main agenda in this paper is interval forecasts and their evaluation. Such forecasts are usually bounded by two quantile forecasts. However, a limitation of quantiles is that they convey no information regarding the size of potential exceedances. By contrast, the location of an expectile is dictated by the whole distribution. This prompts us to propose expectile-bounded intervals. We provide interpretation, a consistent scoring function and a calibration test. Before doing this, we reflect on the evaluation of forecasts of quantile-bounded intervals and expectiles, and suggest extensions of previously proposed calibration tests in order to guard against strategic forecasting. We illustrate ideas using day-ahead electricity price forecasting.

[1]  James W. Taylor.  A strategic predictive distribution for tests of probabilistic calibration , 2020 .

[2]  L. Glosten,et al.  On the Relation between the Expected Value and the Volatility of the Nominal Excess Return on Stocks , 1993 .

[3]  Evangelos Spiliotis,et al.  The M4 Competition: Results, findings, conclusion and way forward , 2018, International Journal of Forecasting.

[4]  W. Newey,et al.  Asymmetric Least Squares Estimation and Testing , 1987 .

[5]  R. Gerlach,et al.  Semi-parametric dynamic asymmetric Laplace models for tail risk forecasting, incorporating realized measures , 2018, International Journal of Forecasting.

[6]  James W. Taylor.  Evaluating Volatility and Interval Forecasts , 1999 .

[7]  H. Tong,et al.  Asymmetric least squares regression estimation: A nonparametric approach ∗ , 1996 .

[8]  T. Gneiting.  Making and Evaluating Point Forecasts , 2009, 0912.0902.

[9]  R. Weron,et al.  Forecasting spot electricity prices: A comparison of parametric and semiparametric time series models , 2008 .

[10]  Peter Christoffersen,et al.  Elements of Financial Risk Management , 2003 .

[11]  Yael Grushka-Cockayne,et al.  The Wisdom of Competitive Crowds , 2013 .

[12]  Peter F. Christoffersen.  Evaluating Interval Forecasts , 1998 .

[13]  R. Weron,et al.  Recent advances in electricity price forecasting: A review of probabilistic forecasting , 2016 .

[14]  Christopher R. Knittel,et al.  An empirical examination of restructured electricity prices , 2005 .

[15]  Chung-Ming Kuan,et al.  Assessing Value at Risk With CARE, the Conditional Autoregressive Expectile Models , 2008 .

[16]  Wojciech Olszewski,et al.  Calibration and Expert Testing , 2015 .

[17]  James W. Taylor.  Estimating Value at Risk and Expected Shortfall Using Expectiles , 2007 .

[18]  Halbert White,et al.  Tests of Conditional Predictive Ability , 2003 .

[19]  Johanna F. Ziegel,et al.  Elicitability and backtesting: Perspectives for banking regulation , 2016, 1608.05498.

[20]  R. L. Winkler.  A Decision-Theoretic Approach to Interval Estimation , 1972 .

[21]  Johanna F. Ziegel,et al.  COHERENCE AND ELICITABILITY , 2013, 1303.1690.

[22]  Florian Ziel,et al.  Variance Stabilizing Transformations for Electricity Spot Price Forecasting , 2018, IEEE Transactions on Power Systems.

[23]  A. Raftery,et al.  Strictly Proper Scoring Rules, Prediction, and Estimation , 2007 .

[24]  Philippe Artzner,et al.  Coherent Measures of Risk , 1999 .

[25]  R. Koenker,et al.  Goodness of Fit and Related Inference Processes for Quantile Regression , 1999 .