Adaptive transfer learning in deep neural networks: Wind power prediction using knowledge transfer from region to region and between different task domains

Transfer learning (TL) in deep neural networks is gaining importance because, in most of the applications, the labeling of data is costly and time consuming. Additionally, TL also provides an effective weight initialization strategy for deep neural networks. This paper introduces the idea of adaptive TL in deep neural networks (ATL‐DNN) for wind power prediction. Specifically, we show in case of wind power prediction that adaptive TL of the deep neural networks system can be adaptively modified as regards training on a different wind farm is concerned. The proposed ATL‐DNN technique is tested for short‐term wind power prediction, where continuously arriving information has to be exploited. Adaptive TL not only helps in providing good weight initialization, but also in utilizing the incoming data for effective learning. Additionally, the proposed ATL‐DNN technique is shown to transfer knowledge between different task domains (wind power to wind speed prediction) and from one region to another region. The simulation results show that the proposed ATL‐DNN technique achieves average values of 0.0637, 0.0986, and 0.0984 for the mean absolute error, root mean squared error, and standard deviation error, respectively.

[1]  Jing Shi,et al.  Fine tuning support vector machines for short-term wind speed forecasting , 2011 .

[2]  Asifullah Khan,et al.  A New Channel Boosted Convolution Neural Network using Transfer Learning , 2018, ArXiv.

[3]  Yuan Yue,et al.  Research of Wind Power Prediction Model Based on RBF Neural Network , 2013, International Conference Communication and Information Systems.

[4]  H Zareipour,et al.  Wind Power Prediction by a New Forecast Engine Composed of Modified Hybrid Neural Network and Enhanced Particle Swarm Optimization , 2011, IEEE Transactions on Sustainable Energy.

[5]  Xinxin Zhu,et al.  Short-Term Spatio-Temporal Wind Power Forecast in Robust Look-ahead Power System Dispatch , 2014, IEEE Transactions on Smart Grid.

[6]  Ming Liu,et al.  A Rolling Grey Model Optimized by Particle Swarm Optimization in Economic Prediction , 2016, Comput. Intell..

[7]  Hojjat Adeli,et al.  Biomedical Applications Based on Natural and Artificial Computing , 2017, Lecture Notes in Computer Science.

[8]  M. Negnevitsky,et al.  Short term wind power forecasting using adaptive neuro-fuzzy inference systems , 2007, 2007 Australasian Universities Power Engineering Conference.

[9]  Honglak Lee,et al.  Adaptive Multi-Column Deep Neural Networks with Application to Robust Image Denoising , 2013, NIPS.

[10]  Andrew Kusiak,et al.  Optimization of wind turbine energy and power factor with an evolutionary computation algorithm , 2010 .

[11]  Geoffrey E. Hinton A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[12]  Asifullah Khan,et al.  Two-phase deep convolutional neural network for reducing class skewness in histopathological images based breast cancer detection , 2017, Comput. Biol. Medicine.

[13]  Sanchit Gupta,et al.  Transfer Learning by Finetuning Pretrained CNNs Entirely with Synthetic Images , 2017, NCVPRIPG.

[14]  Maria Grazia De Giorgi,et al.  Error analysis of short term wind power prediction models , 2011 .

[15]  이상헌,et al.  Deep Belief Networks , 2010, Encyclopedia of Machine Learning.

[16]  Qinghua Hu,et al.  Transfer learning for short-term wind speed prediction with deep neural networks , 2016 .

[17]  Muhammad Hanif Durad,et al.  Intrusion detection using deep sparse auto-encoder and self-taught learning , 2019, Neural Computing and Applications.

[18]  S. N. Singh,et al.  AWNN-Assisted Wind Power Forecasting Using Feed-Forward Neural Network , 2012, IEEE Transactions on Sustainable Energy.

[19]  Taher Niknam,et al.  Probabilistic wind power forecasting using a novel hybrid intelligent method , 2016, Neural Computing and Applications.

[20]  Xiaofeng Yang,et al.  A hybrid strategy of short term wind power prediction , 2013 .

[21]  Yi Wu,et al.  A general framework for transfer sparse subspace learning , 2012, Neural Computing and Applications.

[22]  Paris Smaragdis,et al.  Adaptive Denoising Autoencoders: A Fine-Tuning Scheme to Learn from Test Mixtures , 2015, LVA/ICA.

[23]  P. K. Dash,et al.  Short-term prediction of wind power using a hybrid pseudo-inverse Legendre neural network and adaptive firefly algorithm , 2017, Neural Computing and Applications.

[24]  Jianzhou Wang,et al.  Forecasting wind speed using empirical mode decomposition and Elman neural network , 2014, Appl. Soft Comput..

[25]  Joao P. S. Catalao,et al.  Short-term wind power forecasting using adaptive neuro-fuzzy inference system combined with evolutionary particle swarm optimization, wavelet transform and mutual information , 2015 .

[26]  Pietro Vecchio,et al.  Wind energy prediction using a two-hidden layer neural network , 2010 .

[27]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[28]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[29]  Yifan Gong,et al.  Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[30]  Peyman Abbaszadeh,et al.  Development of a coupled wavelet transform and evolutionary Levenberg‐Marquardt neural networks for hydrological process modeling , 2018, Comput. Intell..

[31]  K. Kulat,et al.  Pattern sequence similarity based techniques for wind speed forecasting , 2017 .

[32]  Shahaboddin Shamshirband,et al.  Predicting the wind power density based upon extreme learning machine , 2015 .

[33]  Sven Behnke,et al.  Deep Learning , 2012, KI - Künstliche Intelligenz.

[34]  Ekaterina Mangalova,et al.  Wind power forecasting using the k-nearest neighbors algorithm , 2014 .

[35]  Tauseef Jamal,et al.  Deep Belief Networks Based Feature Generation and Regression for Predicting Wind Power , 2018, ArXiv.

[36]  Lars Landberg,et al.  Short-term prediction of the power production from wind farms , 1999 .

[37]  Asifullah Khan,et al.  Wind power prediction using deep neural network based meta regression and transfer learning , 2017, Appl. Soft Comput..

[38]  Conversion and delivery of electrical energy in the 21st century , 2008, 2008 IEEE Power and Energy Society General Meeting - Conversion and Delivery of Electrical Energy in the 21st Century.

[39]  Shigeru Katagiri,et al.  Speaker Adaptive Training using Deep Neural Networks , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  Yongqian Liu,et al.  Short-Term Wind-Power Prediction Based on Wavelet Transform–Support Vector Machine and Statistic-Characteristics Analysis , 2012, IEEE Transactions on Industry Applications.

[41]  Asifullah Khan,et al.  Intelligent and robust prediction of short term wind power using genetic programming based ensemble of neural networks , 2017 .

[42]  Hamidreza Zareipour,et al.  Wind power forecast using wavelet neural network trained by improved Clonal selection algorithm , 2015 .

[43]  Paras Mandal,et al.  A Hybrid Intelligent Model for Deterministic and Quantile Regression Approach for Probabilistic Wind Power Forecasting , 2014, IEEE Transactions on Power Systems.

[44]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[45]  Alicia Troncoso Lora,et al.  Deep Learning-Based Approach for Time Series Forecasting with Application to Electricity Load , 2017, IWINAC.

[46]  Asifullah Khan,et al.  A survey of the recent architectures of deep convolutional neural networks , 2019, Artificial Intelligence Review.

[47]  F. Cassola,et al.  Wind speed and wind energy forecast through Kalman filtering of Numerical Weather Prediction model output , 2012 .

[48]  Chee Peng Lim,et al.  Transfer learning using the online Fuzzy Min–Max neural network , 2013, Neural Computing and Applications.

[49]  U. Focken,et al.  New developments in wind energy forecasting , 2008, 2008 IEEE Power and Energy Society General Meeting - Conversion and Delivery of Electrical Energy in the 21st Century.

[50]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[51]  Nima Amjady,et al.  Solar energy forecasting based on hybrid neural network and improved metaheuristic algorithm , 2018, Comput. Intell..

[52]  Peng Guo,et al.  A Review of Wind Power Forecasting Models , 2011 .