A PSO Boosted Ensemble of Extreme Learning Machines for Time Series Forecasting

In this work, a first approach of using the Particle Swarm Optimization (PSO) as a method for optimizing an Ensemble Model built with Extreme Learning Machines is presented. The paper focuses on the obtaining of the parameters of a weighted averaging method for a Ensemble Model, using Extreme Learning Machines as models. The main contribution of this document is the use of the heuristic algorithm PSO for searching optimum parameters of the weighted averaging method. The experiments show that PSO is suitable for computing the parameters of the ensemble, obtaining an average improvement of 68% of the error comparing with an individual model. Also other comparisons have been made with basic combining methods of Ensemble Model fulfilling the expectations.

[1]  Guoqiang Peter Zhang,et al.  Time series forecasting using a hybrid ARIMA and neural network model , 2003, Neurocomputing.

[2]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[3]  S. A. Jafari,et al.  Robust combining methods in committee neural networks , 2011, 2011 IEEE Symposium on Computers & Informatics.

[4]  Thomas G. Dietterich.  Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[5]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Wei Fan,et al.  An adaptive ensemble model of extreme learning machine for time series prediction , 2015, 2015 12th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP).

[7]  Maciej Ogorzalek,et al.  Time series prediction with ensemble models applied to the CATS benchmark , 2007, Neurocomputing.

[8]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[9]  Oscar Castillo,et al.  Particle Swarm Optimization of the Fuzzy Integrators for Time Series Prediction Using Ensemble of IT2FNN Architectures , 2017, Nature-Inspired Design of Hybrid Intelligent Systems.

[10]  M. Ogorzalek,et al.  Time series prediction with ensemble models , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[11]  A. S. Weigend,et al.  Results of the time series prediction competition at the Santa Fe Institute , 1993, IEEE International Conference on Neural Networks.

[12]  David W. Opitz,et al.  Actively Searching for an E(cid:11)ective Neural-Network Ensemble , 1996 .

[13]  Ram Gopal Raj,et al.  The artificial neural network for solar radiation prediction and designing solar systems: a systematic literature review , 2015 .

[14]  Shuichi Kurogi,et al.  Time series prediction of the CATS benchmark using Fourier bandpass filters and competitive associative nets , 2007, Neurocomputing.

[15]  Milton S. Boyd,et al.  Designing a neural network for forecasting financial and economic time series , 1996, Neurocomputing.

[16]  Oscar Castillo,et al.  A new approach for time series prediction using ensembles of ANFIS models with interval type-2 and type-1 fuzzy integrators , 2013, 2013 IEEE Conference on Computational Intelligence for Financial Engineering & Economics (CIFEr).

[17]  Saso Dzeroski,et al.  Ensembles of Multi-Objective Decision Trees , 2007, ECML.

[18]  Andries Petrus Engelbrecht,et al.  A study of particle swarm optimization particle trajectories , 2006, Inf. Sci..

[19]  Voratas Kachitvichyanukul,et al.  Comparison of Three Evolutionary Algorithms: GA, PSO, and DE , 2012 .

[20]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[21]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[22]  Eloy Irigoyen,et al.  Gas Consumption Prediction Based on Artificial Neural Networks for Residential Sectors , 2017, SOCO-CISIS-ICEUTE.

[23]  Zhang-Quan Shen,et al.  Optimizing Weights by Genetic Algorithm for Neural Network Ensemble , 2004, ISNN.