A New Approach for Time Series Prediction Using Ensembles of IT2FNN Models with Optimization of Fuzzy Integrators

This paper describes a new approach for time series prediction based on using different soft computing techniques, such as neural networks (NNs), type-1 and type-2 fuzzy logic systems and bio-inspired algorithms, where each of these intelligent techniques can provide a variety of features for solving real and complex problems. Therefore, this paper describes the application of ensembles of interval type-2 fuzzy neural network (IT2FNN) models. The IT2FNN uses hybrid learning algorithm techniques from NNs models and fuzzy logic systems. The output of the Ensemble of IT2FNN models needs the integration process to forecast the time series, and we are required to design the fuzzy integrator (FI) to solve this real problem. Genetic algorithms and particle swarm optimization are used for the optimization of the parameter values in the membership functions of the FI. We consider different time series to measure the performance of the proposed model, and these time series are: Mackey–Glass, Mexican Stock Exchange (MSE or BMV), Dow Jones and NASDAQ. The forecasting errors are calculated as follows: mean absolute error, mean square error (MSE), root-mean-square error, mean percentage error and mean absolute percentage error. The best prediction errors are illustrated as follows: 0.00025 for the Mackey–Glass, 0.01012 for the MSE, 0.01307 for the Dow Jones and 0.01171 for the NASDAQ time series. Simulation results are compared using a statistical test and provide evidence of the potential advantages of the proposed approach.

[1]  Witold Pedrycz,et al.  Concepts and Design Aspects of Granular Models of Type-1 and Type-2 , 2015, Int. J. Fuzzy Log. Intell. Syst..

[2]  Witold Pedrycz,et al.  Fuzzy evolutionary computation , 1997 .

[3]  Oscar Castillo,et al.  Optimization of type-2 fuzzy systems based on bio-inspired methods: A concise review , 2012, Inf. Sci..

[4]  N. N. Karnik,et al.  Introduction to type-2 fuzzy logic systems , 1998, 1998 IEEE International Conference on Fuzzy Systems Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36228).

[5]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[6]  Mohammad Valipour,et al.  Selecting the best model to estimate potential evapotranspiration with respect to climate change and magnitudes of extreme events , 2017 .

[7]  M. Valipour,et al.  Comparison of the ARMA, ARIMA, and the autoregressive artificial neural network models in forecasting the monthly inflow of Dez dam reservoir , 2013 .

[8]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[9]  Oscar Castillo,et al.  Interval type-2 fuzzy weight adjustment for backpropagation neural networks with application in time series prediction , 2014, Inf. Sci..

[10]  Piero P. Bonissone,et al.  Evolutionary algorithms + domain knowledge = real-world evolutionary computation , 2006, IEEE Transactions on Evolutionary Computation.

[11]  Jerry M. Mendel,et al.  Type-2 fuzzy logic systems , 1999, IEEE Trans. Fuzzy Syst..

[12]  Oscar Castillo,et al.  Comparison of Hybrid Intelligent Systems, Neural Networks and Interval Type-2 Fuzzy Logic for Time Series Prediction , 2007, 2007 International Joint Conference on Neural Networks.

[13]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[14]  P. Cowpertwait,et al.  Introductory Time Series with R , 2009 .

[15]  Daniela Panno,et al.  An integrated fuzzy-GA approach for buffer management , 2006, IEEE Transactions on Fuzzy Systems.

[16]  Stella E. Okello,et al.  Farmer’s Knowledge and Perceptions on Rice Insect Pests and Their Management in Uganda , 2016 .

[17]  Sara G. Castellanos,et al.  Development of the Mexican Bond Market , 2008 .

[18]  Ebrahim H. Mamdani,et al.  An Experiment in Linguistic Synthesis with a Fuzzy Logic Controller , 1999, Int. J. Hum. Comput. Stud..

[19]  Mohammad Valipour,et al.  How Much Meteorological Information Is Necessary to Achieve Reliable Accuracy for Rainfall Estimations , 2016 .

[20]  Dongrui Wu,et al.  Genetic learning and performance evaluation of interval type-2 fuzzy logic controllers , 2006, Eng. Appl. Artif. Intell..

[21]  Chi-Hsu Wang,et al.  Dynamical optimal training for interval type-2 fuzzy neural network (T2FNN) , 2003, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[22]  Oscar Castillo,et al.  Interval Type-2 Fuzzy Logic Toolbox , 2007, Eng. Lett..

[23]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[24]  M. Sugeno,et al.  Derivation of Fuzzy Control Rules from Human Operator's Control Actions , 1983 .

[25]  Lin Peng,et al.  A Call Auction’s Impact on Price Formation and Order Routing: Evidence from the NASDAQ Stock Market , 2012 .

[26]  Patricia Melin,et al.  A New Method for Type-2 Fuzzy Integration in Ensemble Neural Networks Based on Genetic Algorithms , 2013, Recent Advances on Hybrid Intelligent Systems.

[27]  Shu-Xian Lun,et al.  A novel model of leaky integrator echo state network for time-series prediction , 2015, Neurocomputing.

[28]  William W. S. Wei,et al.  Time series analysis - univariate and multivariate methods , 1989 .

[29]  J. Mendel.  Uncertain Rule-Based Fuzzy Logic Systems: Introduction and New Directions , 2001 .

[30]  Lawrence W. Lan,et al.  Genetic fuzzy logic controller: an iterative evolution algorithm with new encoding method , 2005, Fuzzy Sets Syst..

[31]  Lotfi A. Zadeh,et al.  Fuzzy logic, neural networks, and soft computing , 1993, CACM.

[32]  Yoshiki Uchikawa,et al.  On fuzzy modeling using fuzzy neural networks with the back-propagation algorithm , 1992, IEEE Trans. Neural Networks.

[33]  Frederick E. Petry,et al.  Genetic Algorithms , 1992 .

[34]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[35]  Ola Hössjer,et al.  Multivariate Time Series Modeling, Estimation and Prediction of Mortalities , 2015 .

[36]  Oscar Castillo,et al.  A Hybrid Learning Algorithm for Interval Type-2 Fuzzy Neural Networks: The Case of Time Series Prediction , 2008, Soft Computing for Hybrid Intelligent Systems.

[37]  Jerry M. Mendel,et al.  Applications of Type-2 Fuzzy Logic Systems to Forecasting of Time-series , 1999, Inf. Sci..

[38]  M. W. A. Smith,et al.  Corrections to the paper "the identification of the parameters of time-invariant stochastic systems by a method derived from the continuous-time kalman filter" , 1980, Inf. Sci..

[39]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[40]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[41]  Yu-Ching Lin,et al.  Type-2 Fuzzy Neuro System Via Input-to-State-Stability Approach , 2007, ISNN.

[42]  Patricia Melin,et al.  Optimization of Ensemble Neural Networks with Fuzzy Integration Using the Particle Swarm Algorithm for Time Series Prediction , 2015, Design of Intelligent Systems Based on Fuzzy Logic, Neural Networks and Nature-Inspired Optimization.

[43]  Siem Jan Koopman,et al.  Time Series Analysis by State Space Methods , 2001 .

[44]  Arun Agarwal,et al.  Recurrent neural network and a hybrid model for prediction of stock returns , 2015, Expert Syst. Appl..

[45]  Witold Pedrycz.  Fuzzy Modelling: Paradigms and Practice , 2011 .

[46]  Hisao Ishibuchi,et al.  Selecting fuzzy if-then rules for classification problems using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[47]  Kalyanmoy Deb,et al.  A population-based algorithm-generator for real-parameter optimization , 2005, Soft Comput..

[48]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[49]  Mohammad Valipour.  Ability of Box-Jenkins Models to Estimate of Reference Potential Evapotranspiration (A Case Study: Mehrabad Synoptic Station, Tehran, Iran) , 2012 .

[50]  Jerry M. Mendel,et al.  A vector similarity measure for linguistic approximation: Interval type-2 and type-1 fuzzy sets , 2008, Inf. Sci..

[51]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[52]  Hugo Jair Escalante,et al.  Particle Swarm Model Selection , 2009, J. Mach. Learn. Res..

[53]  Martin T. Hagan,et al.  Neural network design , 1995 .

[54]  R. J. S. Salgado,et al.  Volatility dependence structure between the Mexican Stock Exchange and the World Capital Market , 2015 .

[55]  Yu-Ching Lin,et al.  System Identification and Adaptive Filter Using a Novel Fuzzy Neuro System , 2007 .

[56]  Lotfi A. Zadeh,et al.  Fuzzy logic = computing with words , 1996, IEEE Trans. Fuzzy Syst..

[57]  Jee-Hyong Lee,et al.  Constructing Efficient Regional Hazardous Weather Prediction Models through Big Data Analysis , 2016, Int. J. Fuzzy Log. Intell. Syst..

[58]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[59]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[60]  Jyh-Shing Roger Jang,et al.  Fuzzy Modeling Using Generalized Neural Networks and Kalman Filter Algorithm , 1991, AAAI.

[61]  Benjamin M. Blau,et al.  Information in Short Selling: Comparing NASDAQ and the NYSE , 2009 .

[62]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[63]  Oscar Castillo,et al.  A new approach for time series prediction using ensembles of ANFIS models , 2012, Expert Syst. Appl..

[64]  Hani Hagras.  Comments on "Dynamical Optimal Training for Interval Type-2 Fuzzy Neural Network (T2FNN) , 2006, IEEE Trans. Syst. Man Cybern. Part B.

[65]  Martha Pulido,et al.  Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange , 2014, Inf. Sci..

[66]  Witold Pedrycz,et al.  Time series long-term forecasting model based on information granules and fuzzy clustering , 2015, Eng. Appl. Artif. Intell..

[67]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[68]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[69]  Mohammad Valipour,et al.  Analysis of potential evapotranspiration using limited weather data , 2017, Applied Water Science.

[70]  Oscar Castillo,et al.  Time series prediction using ensembles of ANFIS models with genetic optimization of interval type-2 and type-1 fuzzy integrators , 2014, Int. J. Hybrid Intell. Syst..

[71]  Oscar Castillo,et al.  Optimization of type-2 fuzzy weight for neural network using genetic algorithm and particle swarm optimization , 2013, 2013 World Congress on Nature and Biologically Inspired Computing.

[72]  J. Sidaoui,et al.  The Mexican financial system: reforms and evolution 1995-2005 , 2006 .

[73]  Myung-Geun Chun,et al.  Post-Chlorination Process Control based on Flow Prediction by Time Series Neural Network in Water Treatment Plant , 2016, Int. J. Fuzzy Log. Intell. Syst..