An overview of performance evaluation metrics for short-term statistical wind power forecasting

Abstract Wind power forecasting has become an essential tool for energy trading and the operation of the grid due to the increasing importance of wind energy. Therefore, estimating the forecast accuracy of a WPF model and understanding how the accuracy is calculated are necessary steps to appropriately validate WPF models. The present study gives an extensive overview of the performance evaluation methods used for assessing the forecast accuracy of short-term statistical wind power forecast estimates, and the concept of robustness is introduced to determine the validity of a model over different wind power generation scenarios over the testing set. Finally, a numerical study using decomposition-based hybrid models is presented to analyse the robustness of the performance evaluation metrics under different conditions in the context of wind power forecasting. Data from Ireland are employed using two different resolutions to examine its influence on the forecast accuracy.

[1]  Pierre Pinson,et al.  The future of forecasting for renewable energy , 2019, WIREs Energy and Environment.

[2]  Pinar Karagoz,et al.  A Novel Wind Power Forecast Model: Statistical Hybrid Wind Power Forecast Technique (SHWIP) , 2015, IEEE Transactions on Industrial Informatics.

[3]  Seref Sagiroglu,et al.  Data mining and wind power prediction: A literature review , 2012 .

[4]  Miao He,et al.  Support-Vector-Machine-Enhanced Markov Model for Short-Term Wind Power Forecast , 2015, IEEE Transactions on Sustainable Energy.

[5]  Duehee Lee,et al.  Short-Term Wind Power Ensemble Prediction Based on Gaussian Processes and Neural Networks , 2014, IEEE Transactions on Smart Grid.

[6]  Pengfei Chen,et al.  An Inter Type-2 FCR Algorithm Based T–S Fuzzy Model for Short-Term Wind Power Interval Prediction , 2019, IEEE Transactions on Industrial Informatics.

[7]  Zhao Yang Dong,et al.  An advanced approach for optimal wind power generation prediction intervals by using self-adaptive evolutionary extreme learning machine , 2018, Renewable Energy.

[8]  Ming Yang,et al.  A Multi-Model Combination Approach for Probabilistic Wind Power Forecasting , 2017, IEEE Transactions on Sustainable Energy.

[9]  C. Willmott.  ON THE VALIDATION OF MODELS , 1981 .

[10]  Rob J Hyndman,et al.  Another look at measures of forecast accuracy , 2006 .

[11]  Henrik Madsen,et al.  A review on the young history of the wind power short-term prediction , 2008 .

[12]  Jianxue Wang,et al.  K-nearest neighbors and a kernel density estimator for GEFCom2014 probabilistic wind power forecasting , 2016 .

[13]  Pierre Pinson,et al.  Online adaptive lasso estimation in vector autoregressive models for high dimensional wind power forecasting , 2019, International Journal of Forecasting.

[14]  Alex J. Cannon.  Quantile regression neural networks: Implementation in R and application to precipitation downscaling , 2011, Comput. Geosci..

[15]  P. Sorensen,et al.  Power Fluctuations From Large Wind Farms , 2007, IEEE Transactions on Power Systems.

[16]  T. Chai,et al.  Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature , 2014 .

[17]  Oliver Kramer,et al.  Machine learning ensembles for wind power prediction , 2016 .

[18]  Qinghua Hu,et al.  Deterministic and probabilistic wind power forecasting using a variational Bayesian-based adaptive robust multi-kernel regression model , 2017 .

[19]  Yonggang Wu,et al.  An Advanced Approach for Construction of Optimal Wind Power Prediction Intervals , 2015, IEEE Transactions on Power Systems.

[20]  Heeyoung Kim,et al.  A new metric of absolute percentage error for intermittent demand forecasts , 2016 .

[21]  Hamidreza Zareipour,et al.  A review and discussion of decomposition-based hybrid models for wind energy forecasting applications , 2019, Applied Energy.

[22]  Rob J Hyndman,et al.  Probabilistic energy forecasting: Global Energy Forecasting Competition 2014 and beyond , 2016 .

[23]  P Pinson,et al.  Conditional Prediction Intervals of Wind Power Generation , 2010, IEEE Transactions on Power Systems.

[24]  Norden E. Huang,et al.  Ensemble Empirical Mode Decomposition: a Noise-Assisted Data Analysis Method , 2009, Adv. Data Sci. Adapt. Anal..

[25]  Abbas Khosravi,et al.  Short-Term Load and Wind Power Forecasting Using Neural Network-Based Prediction Intervals , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[27]  Amir F. Atiya,et al.  A review and comparison of strategies for multi-step ahead time series forecasting based on the NN5 forecasting competition , 2011, Expert Syst. Appl..

[28]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[29]  C. Y. Chung,et al.  A Fuzzy Adaptive Probabilistic Wind Power Prediction Framework Using Diffusion Kernel Density Estimators , 2018, IEEE Transactions on Power Systems.

[30]  Rasool Azimi,et al.  A hybrid wind power forecasting model based on data mining and wavelets analysis , 2016 .

[31]  Chengshi Tian,et al.  A novel two-stage forecasting model based on error factor and ensemble method for multi-step wind power forecasting , 2019, Applied Energy.

[32]  Chunhang Chen,et al.  Robustness properties of some forecasting methods for seasonal time series: A Monte Carlo study☆ , 1997 .

[33]  Serkan Buhan,et al.  Multistage Wind-Electric Power Forecast by Using a Combination of Advanced Statistical Methods , 2015, IEEE Transactions on Industrial Informatics.

[34]  Hamidreza Zareipour,et al.  Wind power forecast using wavelet neural network trained by improved Clonal selection algorithm , 2015 .

[35]  Haiyan Li,et al.  Probability density forecasting of wind power using quantile regression neural network and kernel density estimation , 2018 .

[36]  Ricardo J. Bessa,et al.  On-line quantile regression in the RKHS (Reproducing Kernel Hilbert Space) for operational probabilistic forecasting of wind power , 2016 .

[37]  N. Huang,et al.  The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis , 1998, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[38]  Pierre Pinson,et al.  Standardizing the Performance Evaluation of Short-Term Wind Power Prediction Models , 2005 .

[39]  P. Pinson,et al.  Trading Wind Generation From Short-Term Probabilistic Forecasts of Wind Power , 2007, IEEE Transactions on Power Systems.

[40]  Yu JIANG,et al.  Short-term wind power forecasting using hybrid method based on enhanced boosting algorithm , 2017 .

[41]  Cristobal Gallego-Castillo,et al.  A review on the recent history of wind power ramp forecasting , 2015 .

[42]  Sujit Kumar Dash,et al.  Short Term Wind Power Forecasting using Hybrid Variational Mode Decomposition and Multi-Kernel Regularized Pseudo Inverse Neural Network , 2018 .

[43]  R. J. Bessa,et al.  Reserve Setting and Steady-State Security Assessment Using Wind Power Uncertainty Forecast: A Case Study , 2012, IEEE Transactions on Sustainable Energy.

[44]  Xiaoru Wang,et al.  A novel hybrid methodology for short-term wind power forecasting based on adaptive neuro-fuzzy inference system , 2017 .

[45]  Stefano Alessandrini,et al.  A novel application of an analog ensemble for short-term wind power forecasting , 2015 .

[46]  R. L. Winkler.  A Decision-Theoretic Approach to Interval Estimation , 1972 .

[47]  Jianzhou Wang,et al.  Analysis and application of forecasting models in wind power integration: A review of multi-step-ahead wind speed forecasting models , 2016 .

[48]  Jianxue Wang,et al.  Review on probabilistic forecasting of wind power generation , 2014 .

[49]  C. Willmott,et al.  A refined index of model performance , 2012 .

[50]  S. Nahavandi,et al.  Prediction Intervals for Short-Term Wind Farm Power Generation Forecasts , 2013, IEEE Transactions on Sustainable Energy.

[51]  John Bjørnar Bremnes,et al.  Probabilistic wind power forecasts using local quantile regression , 2004 .

[52]  Jian Su,et al.  A novel bidirectional mechanism based on time series model for wind power forecasting , 2016 .

[53]  Joao P. S. Catalao,et al.  Short-term wind power forecasting using adaptive neuro-fuzzy inference system combined with evolutionary particle swarm optimization, wavelet transform and mutual information , 2015 .

[54]  Antti Sorjamaa,et al.  Multiple-output modeling for multi-step-ahead time series forecasting , 2010, Neurocomputing.

[55]  Yitao Liu,et al.  Deep learning based ensemble approach for probabilistic wind power forecasting , 2017 .

[56]  P. Pinson,et al.  Very‐short‐term probabilistic forecasting of wind power with generalized logit–normal distributions , 2012 .

[57]  Evangelos Spiliotis,et al.  Statistical and Machine Learning forecasting methods: Concerns and ways forward , 2018, PloS one.

[58]  Asifullah Khan,et al.  Wind power prediction using deep neural network based meta regression and transfer learning , 2017, Appl. Soft Comput..

[59]  Ali Lahouar,et al.  Hour-ahead wind power forecast based on random forests , 2017 .

[60]  Zhi Zhou,et al.  A Nonparametric Bayesian Framework for Short-Term Wind Power Probabilistic Forecast , 2019, IEEE Transactions on Power Systems.

[61]  Paras Mandal,et al.  A review of wind power and wind speed forecasting methods with different time horizons , 2010, North American Power Symposium 2010.

[62]  Aoife Foley,et al.  Current methods and advances in forecasting of wind power generation , 2012 .

[63]  Henrik Madsen,et al.  Using quantile regression to extend an existing wind power forecasting system with probabilistic forecasts , 2006 .

[64]  Taher Niknam,et al.  Probabilistic wind power forecasting using a novel hybrid intelligent method , 2016, Neural Computing and Applications.

[65]  Zhile Yang,et al.  Time series wind power forecasting based on variant Gaussian Process and TLBO , 2016, Neurocomputing.

[66]  Yanbin Yuan,et al.  Wind power prediction using hybrid autoregressive fractionally integrated moving average and least square support vector machine , 2017 .

[67]  Pierre Pinson,et al.  On‐line assessment of prediction risk for wind power production forecasts , 2003 .

[68]  Zheng Yao,et al.  Wind power forecasting based on principle component phase space reconstruction , 2015 .

[69]  Song Li,et al.  Wind Power Forecasting Using Neural Network Ensembles With Feature Selection , 2015, IEEE Transactions on Sustainable Energy.

[70]  Yongqian Liu,et al.  Wind power grouping forecasts and its uncertainty analysis using optimized relevance vector machine , 2013 .

[71]  Paras Mandal,et al.  A Hybrid Intelligent Model for Deterministic and Quantile Regression Approach for Probabilistic Wind Power Forecasting , 2014, IEEE Transactions on Power Systems.

[72]  C. Granger,et al.  AN INTRODUCTION TO LONG‐MEMORY TIME SERIES MODELS AND FRACTIONAL DIFFERENCING , 1980 .

[73]  H. J. Lu,et al.  An improved neural network-based approach for short-term wind speed and power forecast , 2017 .

[74]  Akin Tascikaraoglu,et al.  A review of combined approaches for prediction of short-term wind speed and power , 2014 .

[75]  Jin Lin,et al.  Direct Quantile Regression for Nonparametric Probabilistic Forecasting of Wind Power Generation , 2017, IEEE Transactions on Power Systems.

[76]  Stanislav Mišák,et al.  Direct Wind Power Forecasting Using a Polynomial Decomposition of the General Differential Equation , 2018, IEEE Transactions on Sustainable Energy.

[77]  Bo Gu,et al.  Research on short-term wind power combined forecasting and its Gaussian cloud uncertainty to support the integration of renewables and EVs , 2020, Renewable Energy.

[78]  Jianchun Peng,et al.  A review of deep learning for renewable energy forecasting , 2019, Energy Conversion and Management.

[79]  H. Madsen,et al.  From probabilistic forecasts to statistical scenarios of short-term wind power production , 2009 .

[80]  Dominique Zosso,et al.  Variational Mode Decomposition , 2014, IEEE Transactions on Signal Processing.

[81]  Pierre Pinson,et al.  Non‐parametric probabilistic forecasts of wind power: required properties and evaluation , 2007 .

[82]  Amaury Lendasse,et al.  Long-term prediction of time series by combining direct and MIMO strategies , 2009, 2009 International Joint Conference on Neural Networks.

[83]  Pierre Pinson,et al.  Global Energy Forecasting Competition 2012 , 2014 .

[84]  G. Papaefthymiou,et al.  Modeling of Spatial Dependence in Wind Power Forecast Uncertainty , 2008, Proceedings of the 10th International Conference on Probablistic Methods Applied to Power Systems.

[85]  Venkata Dinavahi,et al.  Direct Interval Forecast of Uncertain Wind Power Based on Recurrent Neural Networks , 2018, IEEE Transactions on Sustainable Energy.

[86]  Yongqian Liu,et al.  Reviews on uncertainty analysis of wind power forecasting , 2015 .

[87]  Gregor Giebel,et al.  Wind power forecasting-a review of the state of the art , 2017 .

[88]  Asifullah Khan,et al.  Intelligent and robust prediction of short term wind power using genetic programming based ensemble of neural networks , 2017 .

[89]  Pierre Pinson,et al.  Very-Short-Term Probabilistic Wind Power Forecasts by Sparse Vector Autoregression , 2016, IEEE Transactions on Smart Grid.

[90]  Saeid Nahavandi,et al.  A New Fuzzy-Based Combined Prediction Interval for Wind Power Forecasting , 2016, IEEE Transactions on Power Systems.

[91]  Amaury Lendasse,et al.  Time series prediction using DirRec strategy , 2006, ESANN.