Assessment of stacked unidirectional and bidirectional long short-term memory networks for electricity load forecasting

Abstract Electricity load forecasting has been a substantial problem in the electric power system management process. An accurate forecasting model is essential to avoid imprecise predictions that can negatively affect system efficiency, economy, and sustainability. Among several prediction techniques, deep learning methods, especially the Long Short-Term Memory (LSTM), have been shown to have a superior performance in predicting the electricity load consumption. However, the consequences of using these methods have not fully been explored in terms of the various hidden layer structures, the depth of the model architecture, and the impact of tuning the model hyperparameters. In this paper, a systematic experimental methodology has been conducted to investigate the impact of using deep-stacked unidirectional (Uni-LSTM) and bidirectional (Bi-LSTM) networks on predicting electricity load consumption. In particular, two stacked configurations, which include two and three LSTM layers, are compared with the single-layered LSTM for both types to show the significant importance of adding the stacked layers. Moreover, for each proposed configuration, a hyperparameter optimization tool has been implemented to obtain the best model. The results indicate that the deep-stacked LSTM layers have no significant improvement in the prediction accuracy; nevertheless, they consume almost twice the time of the single-layered models. Also, the Bi-LSTM networks outperform the Uni-LSTM networks by 76.25%, 75.49%, and 75.35% in terms of Root Mean Square Error (RMSE), with respect to one, two, and three-layer model configurations, respectively. Furthermore, regarding the prediction accuracy comparison over the total tested period, the optimized Bi-LSTM model outperforms both the optimized Uni-LSTM model by 75.98%, 89.1%, and 89.37%, and the Support Vector Regression (SVR) model by 82.54%, 92.59%, and 92.89% in terms of (RMSE), the Mean Average Percentage Error (MAPE), and Mean Absolute Errors (MAE).

[1]  Wenjie Gang,et al.  Assessment of deep recurrent neural network-based strategies for short-term building energy predictions , 2019, Applied Energy.

[2]  Jafar Adabi,et al.  A hierarchical energy management system for multiple home energy hubs in neighborhood grids , 2020 .

[3]  Lyudmila Mihaylova,et al.  Toward efficient energy systems based on natural gas consumption prediction with LSTM Recurrent Neural Networks , 2019, Energy.

[4]  Akin Tascikaraoglu,et al.  A demand side management strategy based on forecasting of residential renewable sources: A smart home system in Turkey , 2014 .

[5]  Sung-Bae Cho,et al.  Predicting residential energy consumption using CNN-LSTM neural networks , 2019, Energy.

[6]  Dan Wang,et al.  Bi-directional long short-term memory method based on attention mechanism and rolling update for short-term load forecasting , 2019, International Journal of Electrical Power & Energy Systems.

[7]  Çagdas Hakan Aladag,et al.  A new linear & nonlinear artificial neural network model for time series forecasting , 2013, Decis. Support Syst..

[8]  Antorweep Chakravorty,et al.  Evaluating Feature Selection Methods for Short-Term Load Forecasting , 2019, 2019 IEEE International Conference on Big Data and Smart Computing (BigComp).

[9]  Jiejin Cai,et al.  Predicting hourly cooling load in the building: A comparison of support vector machine and different artificial neural networks , 2009 .

[10]  Muhammad Hussain,et al.  A review of demand response in an efficient smart grid environment , 2018, The Electricity Journal.

[11]  Kaile Zhou,et al.  Load demand forecasting of residential buildings using a deep learning model , 2020 .

[12]  Bart De Schutter,et al.  Forecasting spot electricity prices Deep learning approaches and empirical comparison of traditional algorithms , 2018 .

[13]  Xu Chen,et al.  A hybrid teaching-learning artificial neural network for building electrical energy consumption prediction , 2018, Energy and Buildings.

[14]  Stefan Lessmann,et al.  A comparative study of LSTM neural networks in forecasting day-ahead global horizontal irradiance with satellite data , 2018 .

[15]  Hui Liu,et al.  Deterministic wind energy forecasting: A review of intelligent predictors and auxiliary methods , 2019, Energy Conversion and Management.

[16]  G. Lightbody,et al.  Smart transactive energy framework in grid-connected multiple home microgrids under independent and coalition operations , 2018, Renewable Energy.

[17]  Hui Qin,et al.  Long Short-Term Memory Network based on Neighborhood Gates for processing complex causality in wind speed prediction , 2019, Energy Conversion and Management.

[18]  Frédéric Magoulès,et al.  A review on the prediction of building energy consumption , 2012 .

[19]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[20]  Yuan Zhang,et al.  Short-Term Residential Load Forecasting Based on LSTM Recurrent Neural Network , 2019, IEEE Transactions on Smart Grid.

[21]  Ran Li,et al.  Deep Learning for Household Load Forecasting—A Novel Pooling Deep RNN , 2018, IEEE Transactions on Smart Grid.

[22]  D. Kirschen,et al.  Economic impact assessment of load forecast errors considering the cost of interruptions , 2006, 2006 IEEE Power Engineering Society General Meeting.

[23]  Liang Chen,et al.  A nonlinear hybrid wind speed forecasting model using LSTM network, hysteretic ELM and Differential Evolution algorithm , 2018, Energy Conversion and Management.

[24]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[25]  Alberto Hernandez Neto,et al.  Comparison between detailed model simulation and artificial neural network for forecasting building energy consumption , 2008 .

[26]  Bart De Schutter,et al.  Forecasting day-ahead electricity prices in Europe: the importance of considering market integration , 2017, ArXiv.

[27]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[28]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[29]  Vivek Srikumar,et al.  Predicting electricity consumption for commercial and residential buildings using deep recurrent neural networks , 2018 .

[30]  Sheng Liu,et al.  Comparative study of data driven methods in building electricity use prediction , 2019 .

[31]  Thomas Fischer,et al.  Deep learning with long short-term memory networks for financial market predictions , 2017, Eur. J. Oper. Res..

[32]  Nora El-Gohary,et al.  A review of data-driven building energy consumption prediction studies , 2018 .

[33]  Svetha Venkatesh,et al.  Exploration Enhanced Expected Improvement for Bayesian Optimization , 2018, ECML/PKDD.

[34]  Rizwan Ahmad,et al.  Intelligent techniques for forecasting electricity consumption of buildings , 2018, Energy.

[35]  Sara Atef,et al.  A Comparative Study Using Deep Learning and Support Vector Regression for Electricity Price Forecasting in Smart Grids , 2019, 2019 IEEE 6th International Conference on Industrial Engineering and Applications (ICIEA).

[36]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[37]  Sara Atef,et al.  Real-Time Load Consumption Prediction and Demand Response Scheme Using Deep Learning in Smart Grids , 2019, 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT).

[38]  Hussein A. Kazem,et al.  Comparison of prediction methods of photovoltaic power system production using a measured dataset , 2017 .

[39]  Hamed H. H. Aly,et al.  A proposed intelligent short-term load forecasting hybrid models of ANN, WNN and KF based on clustering techniques for smart grid , 2020 .

[40]  Xueqing Zhang,et al.  A comprehensive review on the application of artificial neural networks in building energy analysis , 2019, Neurocomputing.

[41]  Federico Silvestro,et al.  Implementation of advanced functionalities for Distribution Management Systems: Load forecasting and modeling through Artificial Neural Networks ensembles , 2019, Electric Power Systems Research.

[42]  Alaa El. Sagheer,et al.  Time series forecasting of petroleum production using deep LSTM recurrent networks , 2019, Neurocomputing.