Conformal prediction interval estimation and applications to day-ahead and intraday power markets

Abstract In this study, we investigated the application of the conformal prediction (CP) concept in the context of short-term electricity price forecasting. In particular, we determined the most important aspects related to the utility of CP, as well as explaining why this simple but highly effective idea has proved useful in other application areas and why its characteristics make it promising for short-term power applications. We compared the performance of CP with various state-of-the-art electricity price forecasting models, such as quantile regression averaging, in an empirical out-of-sample study of three short-term electricity time series. We combined CP with various underlying point forecast models to demonstrate its versatility and behavior under changing conditions. Our findings suggest that CP yields sharp and reliable prediction intervals in short-term power markets. We also inspected the effects of each of the model components to provide path-based guideline regarding how to find the best CP model for each market.

[1]  Ashwani Kumar,et al.  Electricity price forecasting in deregulated markets: A review and evaluation , 2009 .

[2]  Stef van Buuren,et al.  MICE: Multivariate Imputation by Chained Equations in R , 2011 .

[3]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[4]  R. Weron,et al.  Day-ahead electricity price forecasting with high-dimensional structures: Univariate vs. multivariate models , 2016 .

[5]  Haris Haralambous,et al.  Neural Networks Regression Inductive Conformal Predictor and Its Application to Total Electron Content Prediction , 2010, ICANN.

[6]  Jianzhou Wang,et al.  Short-term electricity prices forecasting based on support vector regression and Auto-regressive integrated moving average modeling , 2010 .

[7]  Andy Liaw,et al.  Classification and Regression by randomForest , 2007 .

[8]  Jakub Nowotarski,et al.  A hybrid model for GEFCom2014 probabilistic electricity price forecasting , 2016 .

[9]  Alexander J. Smola,et al.  Support Vector Method for Function Approximation, Regression Estimation and Signal Processing , 1996, NIPS.

[10]  Yunhe Hou,et al.  A New Recursive Dynamic Factor Analysis for Point and Interval Forecast of Electricity Price , 2013, IEEE Transactions on Power Systems.

[11]  R. Weron.  Modeling and Forecasting Electricity Loads and Prices: A Statistical Approach , 2006 .

[12]  R. Weron,et al.  Probabilistic forecasting of electricity spot prices using Factor Quantile Regression Averaging , 2016 .

[13]  D. Dijk,et al.  Forecasting Day-Ahead Electricity Prices: Utilizing Hourly Prices , 2013 .

[14]  Grzegorz Dudek,et al.  Multilayer perceptron for GEFCom2014 probabilistic electricity price forecasting , 2016 .

[15]  Alicia Troncoso Lora,et al.  A Comparison of Two Techniques for Next-Day Electricity Price Forecasting , 2002, IDEAL.

[16]  Vladimir Vovk,et al.  A tutorial on conformal prediction , 2007, J. Mach. Learn. Res..

[17]  Grzegorz Marcjasz,et al.  A Note on Averaging Day-Ahead Electricity Price Forecasts Across Calibration Windows , 2019, IEEE Transactions on Sustainable Energy.

[18]  Saeid Nahavandi,et al.  A neural network-GARCH-based method for construction of Prediction Intervals , 2013 .

[19]  C. Mallows.  More comments on C p , 1995 .

[20]  Jakub Nowotarski,et al.  Computing electricity spot price prediction intervals using quantile regression and forecast averaging , 2015, Comput. Stat..

[21]  Vladimir Vovk,et al.  Cross-conformal predictors , 2012, Annals of Mathematics and Artificial Intelligence.

[22]  Derek W. Bunn,et al.  Analysis and Forecasting of Electricty Price Risks with Quantile Factor Models , 2016 .

[23]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[24]  Jan Dimon Bendtsen,et al.  Heuristic Optimization for the Discrete Virtual Power Plant Dispatch Problem , 2014, IEEE Transactions on Smart Grid.

[25]  Rafał Weron,et al.  Efficient Forecasting of Electricity Spot Prices with Expert and LASSO Models , 2018, Energies.

[26]  Rafał Weron,et al.  Selection of Calibration Windows for Day-Ahead Electricity Price Forecasting , 2018, Energies.

[27]  Henrik Boström,et al.  Regression conformal prediction with random forests , 2014, Machine Learning.

[28]  Trevor Hastie,et al.  Regularization Paths for Generalized Linear Models via Coordinate Descent. , 2010, Journal of statistical software.

[29]  Richard A. Johnson,et al.  A new family of power transformations to improve normality or symmetry , 2000 .

[30]  Michael P. Clements,et al.  Evaluating interval forecasts of high-frequency financial data , 2003 .

[31]  N. Amjady,et al.  Energy price forecasting - problems and proposals for such predictions , 2006 .

[32]  Christoph Weber,et al.  Adequate intraday market design to enable the integration of wind energy into the European power systems , 2010 .

[33]  Florentina Paraschiv,et al.  Econometric Analysis of 15-Minute Intraday Electricity Prices , 2016 .

[34]  Rob J Hyndman,et al.  Probabilistic energy forecasting: Global Energy Forecasting Competition 2014 and beyond , 2016 .

[35]  Florian Ziel,et al.  Variance Stabilizing Transformations for Electricity Spot Price Forecasting , 2018, IEEE Transactions on Power Systems.

[36]  C. L. Mallows.  Some comments on C_p , 1973 .

[37]  Alexander Gammerman,et al.  Learning by Transduction , 1998, UAI.

[38]  R. L. Winkler.  A Decision-Theoretic Approach to Interval Estimation , 1972 .

[39]  W. Gasarch,et al.  The Book Review Column 1 Coverage Untyped Systems Simple Types Recursive Types Higher-order Systems General Impression 3 Organization, and Contents of the Book , 2022 .

[40]  Jakub Nowotarski,et al.  An empirical comparison of alternate schemes for combining electricity spot price forecasts , 2013 .

[41]  David C. Hoaglin,et al.  John W. Tukey and data analysis , 2003 .

[42]  Murray Aitkin,et al.  A general maximum likelihood analysis of overdispersion in generalized linear models , 1996, Stat. Comput..

[43]  Jinliang Zhang,et al.  Day-ahead electricity price forecasting by a new hybrid method , 2012, Comput. Ind. Eng..

[44]  Florentina Paraschiv,et al.  Extended forecast methods for day-ahead electricity spot prices applying artificial neural networks , 2016 .

[45]  R. Weron,et al.  Forecasting spot electricity prices: A comparison of parametric and semiparametric time series models , 2008 .

[46]  Peter F. Christoffersen.  Evaluating Interval Forecasts , 1998 .

[47]  R. Weron,et al.  Recent advances in electricity price forecasting: A review of probabilistic forecasting , 2016 .

[48]  Tao Hong,et al.  Probabilistic Load Forecasting via Quantile Regression Averaging on Sister Forecasts , 2017, IEEE Transactions on Smart Grid.

[49]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[50]  R. Weron.  Electricity price forecasting: A review of the state-of-the-art with a look into the future , 2014 .

[51]  Francesco Lisi,et al.  Combining day-ahead forecasts for British electricity prices , 2013 .

[52]  Lars Carlsson,et al.  Aggregated Conformal Prediction , 2014, AIAI Workshops.

[53]  Jakub Nowotarski,et al.  Merging quantile regression with forecast averaging to obtain more accurate interval forecasts of Nord Pool spot prices , 2014, 11th International Conference on the European Energy Market (EEM14).