Particle swarm optimization for construction of neural network-based prediction intervals

Point forecasts suffer from unreliable and uninformative problems when the uncertainty level increases in data. Prediction intervals (PIs) have been proposed in the literature to quantify uncertainties associated with point forecasts. In this paper, a newly introduced method called Lower Upper Bound Estimation (LUBE) (Khosravi et al., 2011, [1]) is applied and extended for construction of PIs. The LUBE method adopts a neural network (NN) with two outputs to directly generate the upper and lower bounds of PIs without making any assumption about the data distribution. A new width evaluation index that is suitable for NN training is proposed. Further a new cost function is developed for the comprehensive evaluation of PIs based on their width and coverage probability. The width index is replaced by the new one and PSO with mutation operator is used for minimizing the cost function and adjusting NN parameters in the LUBE method. By introducing these two changes we observe dramatic improvements in the quality of results and speed. Demonstrated results for six synthetic and real-world case studies indicate that the proposed PSO-based LUBE method is very efficient in constructing high quality PIs in a short time.

[1]  J. T. Hwang,et al.  Prediction Intervals for Artificial Neural Networks , 1997 .

[2]  Saeid Nahavandi,et al.  Prediction Interval Construction and Optimization for Adaptive Neurofuzzy Inference Systems , 2011, IEEE Transactions on Fuzzy Systems.

[3]  Jong-Cheng Wu,et al.  Prediction of flutter derivatives by artificial neural networks , 2008 .

[4]  Antonio Luchetta.  Automatic generation of the optimum threshold for parameter weighted pruning in multiple heterogeneous output neural networks , 2008, Neurocomputing.

[5]  Dipti Srinivasan,et al.  A SOM-based hybrid linear-neural model for short-term load forecasting , 2011, Neurocomputing.

[6]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[7]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[8]  Michael T. Manry,et al.  An integrated growing-pruning method for feedforward network training , 2008, Neurocomputing.

[9]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Carlos A. Coello Coello,et al.  Handling multiple objectives with particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[11]  Chris Chatfield,et al.  Calculating Interval Forecasts , 1993 .

[12]  Bijaya Ketan Panigrahi,et al.  Adaptive particle swarm optimization approach for static and dynamic economic load dispatch , 2008 .

[13]  Milos Manic,et al.  Neural network architecture selection analysis with application to cryptography location , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[14]  Elisa Ricci,et al.  Improved pruning strategy for radial basis function networks with dynamic decay adjustment , 2006, Neurocomputing.

[15]  Saeid Nahavandi,et al.  Prediction Intervals to Account for Uncertainties in Travel Time Prediction , 2011, IEEE Transactions on Intelligent Transportation Systems.

[16]  Khashayar Khorasani,et al.  New training strategies for constructive neural networks with application to regression problems , 2004, Neural Networks.

[17]  C. Rodriguez,et al.  Energy price forecasting in the Ontario competitive power system market , 2004, IEEE Transactions on Power Systems.

[18]  Saeid Nahavandi,et al.  Construction of Optimal Prediction Intervals for Load Forecasting Problems , 2010, IEEE Transactions on Power Systems.

[19]  Andrew R. Webb,et al.  Statistical Pattern Recognition , 1999 .

[20]  Amir F. Atiya,et al.  Comprehensive Review of Neural Network-Based Prediction Intervals and New Advances , 2011, IEEE Transactions on Neural Networks.

[21]  Tom Heskes,et al.  Practical Confidence and Prediction Intervals , 1996, NIPS.

[22]  J. Vermaak,et al.  Recurrent neural networks for short-term load forecasting , 1998 .

[23]  Bijaya K. Panigrahi,et al.  Hybrid signal processing and machine intelligence techniques for detection, quantification and classification of power quality disturbances , 2009, Eng. Appl. Artif. Intell..

[24]  George Chryssolouris,et al.  Confidence interval prediction for neural network models , 1996, IEEE Trans. Neural Networks.

[25]  Dipti Srinivasan,et al.  Neural Networks for Real-Time Traffic Signal Control , 2006, IEEE Transactions on Intelligent Transportation Systems.

[26]  Raul Cristian Muresan,et al.  Pattern recognition using pulse-coupled neural networks and discrete Fourier transforms , 2003, Neurocomputing.

[27]  Carlos A. Coello Coello,et al.  A constraint-handling mechanism for particle swarm optimization , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[28]  Wei-Jen Lee,et al.  An Integration of ANN Wind Power Estimation Into Unit Commitment Considering the Forecasting Uncertainty , 2007, IEEE Transactions on Industry Applications.

[29]  Amir F. Atiya,et al.  Lower Upper Bound Estimation Method for Construction of Neural Network-Based Prediction Intervals , 2011, IEEE Transactions on Neural Networks.

[30]  Carlos E. Pedreira,et al.  Neural networks for short-term load forecasting: a review and evaluation , 2001 .

[31]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[32]  Bernard Widrow,et al.  Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights , 1990, 1990 IJCNN International Joint Conference on Neural Networks.