Ensemble Residual Networks for Short-Term Load Forecasting

In this paper, we propose a new ensemble residual network model for short-term load forecasting (STLF). This model improves the accuracy of short-term load forecasting (24 hours in advance). The model has a two-stage network structure. First, the different fully-connected layers are combined, and the combined structure is similar to a recurrent neural network (RNN). Features obtained from historical load data are input to the first stage of the model to get preliminary prediction results. The second stage of the model is a modified residual network, and the final predictions are output from here. We use the ensemble snapshot model with learning rate decay to improve the generalization capability of the model. The model proposed in this paper was trained and tested on two public datasets. Numerical testing shows that the proposed model can get better forecasting results in comparison with other methods, and the ensemble method adopted effectively improves the generalization ability of the model.

[1]  Tareq Hossen,et al.  Short-term load forecasting using deep neural networks (DNN) , 2017, 2017 North American Power Symposium (NAPS).

[2]  Agnaldo J. R. Reis,et al.  Feature extraction via multiresolution analysis for short-term load forecasting , 2005, IEEE Transactions on Power Systems.

[3]  Vladimir Ceperic,et al.  A Strategy for Short-Term Load Forecasting by Support Vector Regression Machines , 2013, IEEE Transactions on Power Systems.

[4]  Z.A. Bashir,et al.  Applying Wavelets to Short-Term Load Forecasting Using PSO-Based Neural Networks , 2009, IEEE Transactions on Power Systems.

[5]  Priyanka Singh,et al.  Integration of new evolutionary approach with artificial neural network for solving short term load forecast problem , 2018 .

[6]  Alagan Anpalagan,et al.  Boosted neural networks for improved short-term electric load forecasting , 2017 .

[7]  M. Pantos,et al.  Short-Term Transmission-Loss Forecast for the Slovenian Transmission Power System Based on a Fuzzy-Logic Decision Approach , 2011, IEEE Transactions on Power Systems.

[8]  Miltiadis Alamaniotis,et al.  Evolutionary Multiobjective Optimization of Kernel-Based Very-Short-Term Load Forecasting , 2012, IEEE Transactions on Power Systems.

[9]  Pierluigi Siano,et al.  A Novel RBF Training Algorithm for Short-Term Electric Load Forecasting and Comparative Studies , 2015, IEEE Transactions on Industrial Electronics.

[10]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Song Li,et al.  Short-term load forecasting by wavelet transform and evolutionary extreme learning machine , 2015 .

[12]  Andrew Gordon Wilson,et al.  Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs , 2018, NeurIPS.

[13]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[14]  Xin Yao,et al.  Short-Term Load Forecasting with Neural Network Ensembles: A Comparative Study [Application Notes] , 2011, IEEE Computational Intelligence Magazine.

[15]  Yang Wang,et al.  Short-Term Load-Forecasting Method Based on Wavelet Decomposition With Second-Order Gray Neural Network Model Combined With ADF Test , 2017, IEEE Access.

[16]  R. E. Abdel-Aal.  Improving electric load forecasts using network committees , 2005 .

[17]  Antonello Rizzi,et al.  Short-Term Electric Load Forecasting Using Echo State Networks and PCA Decomposition , 2015, IEEE Access.

[18]  Ying Chen,et al.  Short-Term Load Forecasting: Similar Day-Based Wavelet Neural Networks , 2010, IEEE Transactions on Power Systems.

[19]  Yuexing Peng,et al.  Enhanced Deep Networks for Short-Term and Medium-Term Load Forecasting , 2019, IEEE Access.

[20]  Xiuwen Yi,et al.  DNN-based prediction model for spatio-temporal data , 2016, SIGSPATIAL/GIS.

[21]  Durga Toshniwal,et al.  Empirical Mode Decomposition Based Deep Learning for Electricity Demand Forecasting , 2018, IEEE Access.

[22]  R. Buizza,et al.  Neural Network Load Forecasting with Weather Ensemble Predictions , 2002, IEEE Power Engineering Review.

[23]  Chao-Ming Huang,et al.  Analysis of an adaptive time-series autoregressive moving-average (ARMA) model for short-term load forecasting , 1995 .

[24]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Yuan Zhang,et al.  Short-Term Residential Load Forecasting Based on LSTM Recurrent Neural Network , 2019, IEEE Transactions on Smart Grid.

[26]  Hao Yu,et al.  An Incremental Design of Radial Basis Function Networks , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Xin Yao,et al.  Short-term load forecasting with neural network ensembles: A comparative study , 2011 .

[28]  Alagan Anpalagan,et al.  Improved short-term load forecasting using bagged neural networks , 2015 .

[29]  Sung-Kwan Joo,et al.  Holiday Load Forecasting Using Fuzzy Polynomial Regression With Weather Feature Selection and Adjustment , 2012, IEEE Transactions on Power Systems.

[30]  Zhuofu Deng,et al.  Multi-Scale Convolutional Neural Network With Time-Cognition for Multi-Step Short-Term Load Forecasting , 2019, IEEE Access.

[31]  S. Fan,et al.  Short-term load forecasting based on an adaptive hybrid method , 2006, IEEE Transactions on Power Systems.

[32]  Song Li,et al.  An ensemble approach for short-term load forecasting by extreme learning machine , 2016 .

[33]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[34]  Maciej Filocha,et al.  A hybrid system for forecasting 24-h power load profile for Polish electric grid , 2017, Appl. Soft Comput..

[35]  Hui He,et al.  Short-Term Non-Residential Load Forecasting Based on Multiple Sequences LSTM Recurrent Neural Network , 2018, IEEE Access.

[36]  Kilian Q. Weinberger,et al.  Snapshot Ensembles: Train 1, get M for free , 2017, ICLR.

[37]  Zhongyi Hu,et al.  Comprehensive learning particle swarm optimization based memetic algorithm for model selection in short-term load forecasting using support vector regression , 2014, Appl. Soft Comput..

[38]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[39]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[40]  Alagan Anpalagan,et al.  Joint bagged-boosted artificial neural networks: Using ensemble machine learning to improve short-term electricity load forecasting , 2020 .

[41]  Ali Deihimi,et al.  Application of echo state networks in short-term electric load forecasting , 2012 .

[42]  Farshid Keynia,et al.  Short-term load forecasting of power systems by combination of wavelet transform and neuro-evolutionary algorithm , 2009 .

[43]  Tara N. Sainath,et al.  Improving deep neural networks for LVCSR using rectified linear units and dropout , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[44]  Nawaraj Kumar Mahato,et al.  Research on Short-Term Load Prediction Based on Seq2seq Model , 2019, Energies.

[45]  Jun Hu,et al.  Short-Term Load Forecasting With Deep Residual Networks , 2018, IEEE Transactions on Smart Grid.

[46]  Hassan Soltan,et al.  A methodology for Electric Power Load Forecasting , 2011 .

[47]  Pragya Dwivedi,et al.  A novel hybrid model based on neural network and multi-objective optimization for effective load forecast , 2019, Energy.