Wind Power Forecasting Using Attention-Based Recurrent Neural Networks: A Comparative Study

Wind power is one of the most efficient renewable resources without emissions. Nonetheless, it is difficult to exactly forecast wind power generation given historical power and wind speed information, the failure of which may cost the risk of large-scale outages. This article takes a close look at the artificial recurrent neural network framework in the application of wind power forecasting. More intelligent mechanisms using attention to capture spatial-temporal patterns within historical data are emphasized in this work and are shown to be state-of-the-art for short-term wind power forecasting. Our experiments at a wind farm in southeast Australia using only the historical wind power generation and wind speed records from ambient weather stations show that, e.g., 7.4750% in mean absolute error (MAE) and 0.3345 in the coefficient of variation in the root mean squared error (CV-RMSE) for half-hour-ahead prediction. To interpret how the three models under consideration—the long- and short-term time-series network (LSTNet), the temporal pattern attention-based long short-term memory (TPA-LSTM) and the dual-stage attention-based recurrent neural network (DA-RNN)—work, we visualize and analyze the details of the models so that further improvement can be made by combining the advantageous components of the models.

[1]  Guoqiang Peter Zhang,et al.  Time series forecasting using a hybrid ARIMA and neural network model , 2003, Neurocomputing.

[2]  Takayuki Osogami,et al.  Nonlinear Dynamic Boltzmann Machines for Time-Series Prediction , 2017, AAAI.

[3]  Laith M. Halabi,et al.  Application of the hybrid ANFIS models for long term wind power density prediction with extrapolation capability , 2018, PloS one.

[4]  P. Seidel,et al.  Multilayer perceptron tumour diagnosis based on chromatography analysis of urinary nucleosides , 2007, Neural Networks.

[5]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[6]  Hao Yin,et al.  Wind speed forecasting based on wavelet packet decomposition and artificial neural networks trained by crisscross optimization algorithm , 2016 .

[7]  N. Huang,et al.  The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis , 1998, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[8]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[9]  Chao Liu,et al.  Short-term prediction of wind power using EMD and chaotic theory , 2012 .

[10]  Dominique Zosso,et al.  Variational Mode Decomposition , 2014, IEEE Transactions on Signal Processing.

[11]  Ali Akbar Abdoos,et al.  A new intelligent method based on combination of VMD and ELM for short term wind power forecasting , 2016, Neurocomputing.

[12]  Hung-yi Lee,et al.  Temporal pattern attention for multivariate time series forecasting , 2018, Machine Learning.

[13]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[14]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[15]  Haiping Wu,et al.  Smart wind speed forecasting using EWT decomposition, GWO evolutionary optimization, RELM learning and IEWT reconstruction , 2018 .

[16]  Muhammad Asif Zahoor Raja,et al.  A novel wavenets long short term memory paradigm for wind power prediction , 2020 .

[17]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[18]  Hamidreza Zareipour,et al.  A review and discussion of decomposition-based hybrid models for wind energy forecasting applications , 2019, Applied Energy.

[19]  Geoffrey E. Hinton,et al.  Learning representations of back-propagation errors , 1986 .

[20]  Aida A. Ferreira,et al.  Enhancing Short-Term Wind Power Forecasting through Multiresolution Analysis and Echo State Networks , 2018 .

[21]  Hui Li,et al.  Short-Term Wind Speed and Output Power Forecasting Based on WT and LSSVM , 2009, 2009 International Conference on Information Engineering and Computer Science.

[22]  Yongqian Liu,et al.  Short-Term Wind-Power Prediction Based on Wavelet Transform–Support Vector Machine and Statistic-Characteristics Analysis , 2012, IEEE Transactions on Industry Applications.

[23]  Cheng Zeng,et al.  Short-term wind power prediction based on spatial model , 2017 .

[24]  Ashu Jain,et al.  Hybrid neural network models for hydrologic time series forecasting , 2007, Appl. Soft Comput..

[25]  Henrik Madsen,et al.  Wind Power Forecasting Based on Echo State Networks and Long Short-Term Memory , 2018 .

[26]  A. Kusiak,et al.  Short-Term Prediction of Wind Farm Power: A Data Mining Approach , 2009, IEEE Transactions on Energy Conversion.

[27]  Yongqian Liu,et al.  Short-term forecasting and uncertainty analysis of wind turbine power based on long short-term memory network and Gaussian mixture model , 2019, Applied Energy.

[28]  Jianzhou Wang,et al.  Wind Speed Forecasting Using a Two-Stage Forecasting System With an Error Correcting and Nonlinear Ensemble Strategy , 2019, IEEE Access.

[29]  Saad Mekhilef,et al.  Long-Term Wind Speed Forecasting and General Pattern Recognition Using Neural Networks , 2014, IEEE Transactions on Sustainable Energy.

[30]  A. S. Dokuz,et al.  Wind power forecasting based on daily wind speed data using machine learning algorithms , 2019, Energy Conversion and Management.

[31]  Yuan Zhao,et al.  A novel hybrid model based on VMD-WT and PCA-BP-RBF neural network for short-term wind speed forecasting , 2019, Energy Conversion and Management.

[32]  Chengshi Tian,et al.  A novel two-stage forecasting model based on error factor and ensemble method for multi-step wind power forecasting , 2019, Applied Energy.

[33]  Weisheng Wang,et al.  Data-Driven Correction Approach to Refine Power Curve of Wind Farm Under Wind Curtailment , 2018, IEEE Transactions on Sustainable Energy.

[34]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[35]  Guokun Lai,et al.  Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks , 2017, SIGIR.

[36]  Les E. Atlas,et al.  Recurrent Networks and NARMA Modeling , 1991, NIPS.

[37]  Ali Lahouar,et al.  Hour-ahead wind power forecast based on random forests , 2017 .

[38]  Jian Su,et al.  A novel bidirectional mechanism based on time series model for wind power forecasting , 2016 .

[39]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[40]  Peng Wang,et al.  Short-term wind power forecasting using the hybrid model of improved variational mode decomposition and Correntropy Long Short -term memory neural network , 2021, Energy.

[41]  Li Han,et al.  Wind power forecast based on improved Long Short Term Memory network , 2019 .