Variance-Covariance Based Weighing for Neural Network Ensembles

Neural network (NN) is a popular artificial intelligence technique for solving complicated problems due to their inherent capabilities. However generalization in NN can be harmed by a number of factors including parameter's initialization, inappropriate network topology and setting parameters of the training process itself. Forecast combinations of NN models have the potential for improved generalization and lower training time. A weighted averaging based on Variance-Covariance method that assigns greater weight to the forecasts producing lower error, instead of equal weights is practiced in this paper. While implementing the method, combination of forecasts is done with all candidate models in one experiment and with the best selected models in another experiment. It is observed during the empirical analysis that forecasting accuracy is improved by combining the best individual NN models. Another finding of this study is that reducing the number of NN models increases the diversity and, hence, accuracy.

[1]  Carlos E. Pedreira,et al.  Neural networks for short-term load forecasting: a review and evaluation , 2001 .

[2]  E. Gonzalez-Romera,et al.  Monthly Electric Energy Demand Forecasting Based on Trend Extraction , 2006, IEEE Transactions on Power Systems.

[3]  Kuriakose Athappilly,et al.  A comparative predictive analysis of neural networks (NNs), nonlinear regression and classification and regression tree (CART) models , 2005, Expert Syst. Appl..

[4]  R. Barzamini,et al.  Short term load forecasting for Iran National Power System and its regions using multi layer perceptron and fuzzy inference systems , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[5]  P. McSharry,et al.  Short-Term Load Forecasting Methods: An Evaluation Based on European Data , 2007, IEEE Transactions on Power Systems.

[6]  Abbas Khosravi,et al.  Load Forecasting Accuracy through Combination of Trimmed Forecasts , 2012, ICONIP.

[7]  Saeid Nahavandi,et al.  Quantifying uncertainties of neural network-based electricity price forecasts , 2013 .

[8]  Saeid Nahavandi,et al.  Load Forecasting and Neural Networks: A Prediction Interval-Based Perspective , 2010 .

[9]  C. Granger,et al.  Forecasting Economic Time Series. , 1988 .

[10]  Guoqiang Peter Zhang,et al.  Time series forecasting using a hybrid ARIMA and neural network model , 2003, Neurocomputing.

[11]  Chao-Ming Huang,et al.  Analysis of an adaptive time-series autoregressive moving-average (ARMA) model for short-term load forecasting , 1995 .

[12]  S. Hassan,et al.  Improving load forecasting accuracy through combination of best forecasts , 2012, 2012 IEEE International Conference on Power System Technology (POWERCON).

[13]  C. Granger,et al.  An introduction to bilinear time series models , 1979 .

[14]  Amit Konar,et al.  Computational Intelligence: Principles, Techniques and Applications , 2005 .

[15]  Guoqiang Peter Zhang,et al.  A neural network ensemble method with jittered training data for time series forecasting , 2007, Inf. Sci..

[16]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[17]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  R. Adapa,et al.  The impacts of temperature forecast uncertainty on Bayesian load forecasting , 1998 .

[19]  Z. Baharudin,et al.  Electricity forecasting for small scale power system using artificial neural network , 2011, 2011 5th International Power Engineering and Optimization Conference.

[20]  D. Srinivasan,et al.  Interval Type-2 Fuzzy Logic Systems for Load Forecasting: A Comparative Study , 2012, IEEE Transactions on Power Systems.

[21]  M. El-Hawary,et al.  Load forecasting via suboptimal seasonal autoregressive models and iteratively reweighted least squares estimation , 1993 .

[22]  S. A. Soliman,et al.  Short-term electric load forecasting based on Kalman filtering algorithm with moving window weather and load model , 2004 .

[23]  Michael K. Andersson,et al.  Bayesian Forecast Combination for VAR Models , 2007 .

[24]  Gao,et al.  Bagging Neural Networks for Predicting Water Consumption , 2005 .

[25]  A. Timmermann.  Chapter 4 Forecast Combinations , 2006 .

[26]  J. M. Bates,et al.  The Combination of Forecasts , 1969 .

[27]  Hung Man Tong,et al.  Threshold models in non-linear time series analysis. Lecture notes in statistics, No.21 , 1983 .

[28]  Thomas G. Dietterich.  Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[29]  Kostas S. Metaxiotis,et al.  Artificial intelligence in short term electric load forecasting: a state-of-the-art survey for the researcher , 2003 .

[30]  Saeid Nahavandi,et al.  Combined Nonparametric Prediction Intervals for Wind Power Generation , 2013, IEEE Transactions on Sustainable Energy.

[31]  J. Gooijer,et al.  Some recent developments in non-linear time series modelling, testing, and forecasting☆ , 1992 .

[32]  S. Nahavandi,et al.  Prediction Intervals for Short-Term Wind Farm Power Generation Forecasts , 2013, IEEE Transactions on Sustainable Energy.

[33]  Mohsen Hayati,et al.  Artificial Neural Network Approach for Short Term Load Forecasting for Illam Region , 2007 .

[34]  Bruce W. Schmeiser,et al.  Improving model accuracy using optimal linear combinations of trained neural networks , 1995, IEEE Trans. Neural Networks.