Interval Type-2 Fuzzy Neural Networks for Chaotic Time Series Prediction: A Concise Overview

Chaotic time series widely exists in nature and society (e.g., meteorology, physics, economics, etc.), which usually exhibits seemingly unpredictable features due to its inherent nonstationary and high complexity. Thankfully, multifarious advanced approaches have been developed to tackle the prediction issues, such as statistical methods, artificial neural networks (ANNs), and support vector machines. Among them, the interval type-2 fuzzy neural network (IT2FNN), which is a synergistic integration of fuzzy logic systems and ANNs, has received wide attention in the field of chaotic time series prediction. This paper begins with the structural features and superiorities of IT2FNN. Moreover, chaotic characters identification and phase-space reconstruction matters for prediction are presented. In addition, we also offer a comprehensive review of state-of-the-art applications of IT2FNN, with an emphasis on chaotic time series prediction and summarize their main contributions as well as some hardware implementations for computation speedup. Finally, this paper trends and extensions of this field, along with an outlook of future challenges are revealed. The primary objective of this paper is to serve as a tutorial or referee for interested researchers to have an overall picture on the current developments and identify their potential research direction to further investigation.

[1]  Robert John,et al.  Tuning of Type-2 Fuzzy Systems by Simulated A nnealing to Predict Time Series , 2011 .

[2]  Chin-Teng Lin,et al.  A Hybrid of Cooperative Particle Swarm Optimization and Cultural Algorithm for Neural Fuzzy Networks and Its Prediction Applications , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[3]  Chee Peng Lim,et al.  Data driven modelling based on Recurrent Interval-Valued Metacognitive Scaffolding Fuzzy Neural Network , 2017, Neurocomputing.

[4]  Ankit Kumar Das,et al.  An Evolving Interval Type-2 Neurofuzzy Inference System and Its Metacognitive Sequential Learning Algorithm , 2015, IEEE Transactions on Fuzzy Systems.

[5]  Mojtaba Ahmadieh Khanesar,et al.  Analysis of the Noise Reduction Property of Type-2 Fuzzy Logic Systems Using a Novel Type-2 Membership Function , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[6]  Chin-Teng Lin,et al.  Identification and Prediction of Dynamic Systems Using an Interactively Recurrent Self-Evolving Fuzzy Neural Network , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[7]  William J. Dally,et al.  The GPU Computing Era , 2010, IEEE Micro.

[8]  Chi-Hsu Wang,et al.  Dynamical optimal training for interval type-2 fuzzy neural network (T2FNN) , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[9]  Oscar Castillo,et al.  Comparison of T-Norms and S-Norms for Interval Type-2 Fuzzy Numbers in Weight Adjustment for Neural Networks , 2017, Inf..

[10]  Oscar Castillo,et al.  An Interval Type-2 Fuzzy Neural Network for Chaotic Time Series Prediction with Cross-Validation and Akaike Test , 2011, Soft Computing for Intelligent Control and Mobile Robotics.

[11]  Qun Niu,et al.  A Hybrid Learning Method for Constructing Compact Rule-Based Fuzzy Models , 2013, IEEE Transactions on Cybernetics.

[12]  Chia-Feng Juang,et al.  Speedup of Learning in Interval Type-2 Neural Fuzzy Systems Through Graphic Processing Units , 2015, IEEE Transactions on Fuzzy Systems.

[13]  Robi Polikar,et al.  Incremental Learning of Concept Drift in Nonstationary Environments , 2011, IEEE Transactions on Neural Networks.

[14]  Patricia Melin,et al.  Optimization of Ensemble Neural Networks with Type-2 Fuzzy Integration of Responses for the Dow Jones Time Series Prediction , 2014, Intell. Autom. Soft Comput..

[15]  C. K. Michael Tse,et al.  A modified quantized kernel least mean square algorithm for prediction of chaotic time series , 2016, Digit. Signal Process..

[16]  Edward T. Lee,et al.  Fuzzy Neural Networks , 1975 .

[17]  Chi-Keong Goh,et al.  Co-evolutionary multi-task learning with predictive recurrence for multi-step chaotic time series prediction , 2017, Neurocomputing.

[18]  Shie-Jue Lee,et al.  Data-Based System Modeling Using a Type-2 Fuzzy Neural Network With a Hybrid Learning Algorithm , 2011, IEEE Transactions on Neural Networks.

[19]  Chia-Feng Juang,et al.  Circuit Implementation of Data-Driven TSK-Type Interval Type-2 Neural Fuzzy System With Online Parameter Tuning Ability , 2017, IEEE Transactions on Industrial Electronics.

[20]  Edson Bim,et al.  A Multilayer Perceptron Controller Applied to the Direct Power Control of a Doubly Fed Induction Generator , 2014, IEEE Transactions on Sustainable Energy.

[21]  Chia-Feng Juang,et al.  Data-Driven Interval Type-2 Neural Fuzzy System With High Learning Accuracy and Improved Model Interpretability , 2013, IEEE Transactions on Cybernetics.

[22]  Jerry M. Mendel,et al.  Comments on "alpha -Plane Representation for Type-2 Fuzzy Sets: Theory and Applications" , 2010, IEEE Trans. Fuzzy Syst..

[23]  Ioannis Andreadis,et al.  Design and implementation of a fuzzy hardware structure for morphological color image processing , 2003, IEEE Trans. Circuits Syst. Video Technol..

[24]  Chia-Feng Juang,et al.  An Interval Type-2 Neural Fuzzy Chip With On-Chip Incremental Learning Ability for Time-Varying Data Sequence Prediction and System Control , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[26]  Jane Labadin,et al.  Feature selection based on mutual information , 2015, 2015 9th International Conference on IT in Asia (CITA).

[27]  Jun Wang,et al.  Chaotic Time Series Prediction Based on a Novel Robust Echo State Network , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[28]  Hani Hagras,et al.  Toward General Type-2 Fuzzy Logic Systems Based on zSlices , 2010, IEEE Transactions on Fuzzy Systems.

[29]  Juan R. Castro,et al.  Hybrid Learning for General Type-2 TSK Fuzzy Logic Systems , 2017, Algorithms.

[30]  Mojtaba Ahmadieh Khanesar,et al.  Optimal design of adaptive type-2 neuro-fuzzy systems: A review , 2016, Appl. Soft Comput..

[31]  Arash Miranian,et al.  Developing a Local Least-Squares Support Vector Machines-Based Neuro-Fuzzy Model for Nonlinear and Chaotic Time Series Prediction , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Nong Ye,et al.  Recent Developments in Chaotic Time Series Analysis , 2003, Int. J. Bifurc. Chaos.

[33]  Seyed Hossein Iranmanesh,et al.  A new hybrid enhanced local linear neuro-fuzzy model based on the optimized singular spectrum analysis and its application for nonlinear and chaotic time series forecasting , 2015, Inf. Sci..

[34]  Chin-Teng Lin,et al.  Efficient Self-Evolving Evolutionary Learning for Neurofuzzy Inference Systems , 2008, IEEE Transactions on Fuzzy Systems.

[35]  Cuntai Guan,et al.  An Evolving Type-2 Neural Fuzzy Inference System , 2010, PRICAI.

[36]  M MendelJerry Comments on α-plane representation for type-2 fuzzy sets , 2010 .

[37]  Mojtaba Ahmadieh Khanesar,et al.  Feedback Error Learning Control of Magnetic Satellites Using Type-2 Fuzzy Neural Networks With Elliptic Membership Functions , 2015, IEEE Transactions on Cybernetics.

[38]  Ensemble Neural Network with Type-1 and Type-2 Fuzzy Integration for Time Series Prediction and Its Optimization with PSO , 2016, Imprecision and Uncertainty in Information Representation and Processing.

[39]  Rohitash Chandra,et al.  Competition and Collaboration in Cooperative Coevolution of Elman Recurrent Neural Networks for Time-Series Prediction , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[40]  Liang-Yu Shyu,et al.  Using wavelet transform and fuzzy neural network for VPC detection from the holter ECG , 2004, IEEE Transactions on Biomedical Engineering.

[41]  Chih-Min Lin,et al.  An Efficient Interval Type-2 Fuzzy CMAC for Chaos Time-Series Prediction and Synchronization , 2014, IEEE Transactions on Cybernetics.

[42]  Min Han,et al.  Adaptive Elastic Echo State Network for Multivariate Time Series Prediction , 2016, IEEE Transactions on Cybernetics.

[43]  Wai Keung Wong,et al.  Sparsely connected neural network-based time series forecasting , 2012, Inf. Sci..

[44]  Mojtaba Ahmadieh Khanesar,et al.  A novel type-2 fuzzy membership function: application to the prediction of noisy data , 2010, 2010 IEEE International Conference on Computational Intelligence for Measurement Systems and Applications.

[45]  Oscar Castillo,et al.  Optimization of type-2 fuzzy weights in backpropagation learning for neural networks using GAs and PSO , 2016, Appl. Soft Comput..

[46]  P. Grassberger,et al.  Characterization of Strange Attractors , 1983 .

[47]  Oscar Castillo,et al.  Optimization of type-2 fuzzy systems based on bio-inspired methods: A concise review , 2012, Inf. Sci..

[48]  Oscar Castillo,et al.  Interval type-2 fuzzy weight adjustment for backpropagation neural networks with application in time series prediction , 2014, Inf. Sci..

[49]  Jianchao Fan,et al.  A Collective Neurodynamic Optimization Approach to Nonnegative Matrix Factorization , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[50]  John Cristian Borges Gamboa,et al.  Deep Learning for Time-Series Analysis , 2017, ArXiv.

[51]  Jerry M. Mendel,et al.  Type-2 Fuzzy Sets , 2017 .

[52]  Cuntai Guan,et al.  eT2FIS: An Evolving Type-2 Neural Fuzzy Inference System , 2013, Inf. Sci..

[53]  Chin-Teng Lin,et al.  Simplified Interval Type-2 Fuzzy Neural Networks , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[54]  Chia-Feng Juang,et al.  An Interval Type-2 Fuzzy-Neural Network With Support-Vector Regression for Noisy Regression Problems , 2010, IEEE Transactions on Fuzzy Systems.

[55]  Chin-Teng Lin,et al.  A TSK-Type-Based Self-Evolving Compensatory Interval Type-2 Fuzzy Neural Network (TSCIT2FNN) and Its Applications , 2014, IEEE Transactions on Industrial Electronics.

[56]  Mahardhika Pratama,et al.  An Incremental Type-2 Meta-Cognitive Extreme Learning Machine , 2017, IEEE Transactions on Cybernetics.

[57]  Yusuke Hioka,et al.  Informative Acoustic Feature Selection to Maximize Mutual Information for Collecting Target Sources , 2017, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[58]  Long Thanh Ngo,et al.  Speedup of Interval Type 2 Fuzzy Logic Systems Based on GPU for Robot Navigation , 2012, Adv. Fuzzy Syst..

[59]  Chia-Feng Juang,et al.  Reduced Interval Type-2 Neural Fuzzy System Using Weighted Bound-Set Boundary Operation for Computation Speedup and Chip Implementation , 2013, IEEE Transactions on Fuzzy Systems.

[60]  N. Sundararajan,et al.  Extended sequential adaptive fuzzy inference system for classification problems , 2011, Evol. Syst..

[61]  Lotfi A. Zadeh,et al.  The Concepts of a Linguistic Variable and its Application to Approximate Reasoning , 1975 .

[62]  Tharam S. Dillon,et al.  Optimized Structure of the Traffic Flow Forecasting Model With a Deep Learning Approach , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[63]  Chia-Feng Juang,et al.  Evolutionary interval type-2 fuzzy systems using continuous ant colony optimization algorithms , 2016, 2016 IEEE 11th Conference on Industrial Electronics and Applications (ICIEA).

[64]  Davide Anguita,et al.  Energy Load Forecasting Using Empirical Mode Decomposition and Support Vector Regression , 2013, IEEE Transactions on Smart Grid.

[65]  R. Buizza,et al.  Wind Power Density Forecasting Using Ensemble Predictions and Time Series Models , 2009, IEEE Transactions on Energy Conversion.

[66]  Jerry M. Mendel,et al.  Computing With Words for Hierarchical Decision Making Applied to Evaluating a Weapon System , 2010, IEEE Transactions on Fuzzy Systems.

[67]  Gao Junji Determination of embedding dimension for phase space based on improved G-P method , 2014 .

[68]  H. Schuster,et al.  Proper choice of the time delay for the analysis of chaotic time series , 1989 .

[69]  M. Valipour,et al.  Comparison of the ARMA, ARIMA, and the autoregressive artificial neural network models in forecasting the monthly inflow of Dez dam reservoir , 2013 .

[70]  D. Kugiumtzis State space reconstruction parameters in the analysis of chaotic time series—the role of the time window length , 1996, comp-gas/9602002.

[71]  Jun Wang,et al.  A Dynamic Feedforward Neural Network Based on Gaussian Particle Swarm Optimization and its Application for Predictive Control , 2011, IEEE Transactions on Neural Networks.

[72]  Martha Pulido,et al.  Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the Mexican Stock Exchange , 2014, Inf. Sci..

[73]  Chia-Feng Juang,et al.  Speedup of Implementing Fuzzy Neural Networks With High-Dimensional Inputs Through Parallel Processing on Graphic Processing Units , 2011, IEEE Transactions on Fuzzy Systems.

[74]  Oscar Castillo,et al.  A review on the applications of type-2 fuzzy logic in classification and pattern recognition , 2013, Expert Syst. Appl..

[75]  Chih-Min Lin,et al.  PSO-Self-Organizing Interval Type-2 Fuzzy Neural Network for Antilock Braking Systems , 2017, Int. J. Fuzzy Syst..

[76]  Ruey S. Tsay,et al.  Analysis of Financial Time Series , 2005 .

[77]  Chia-Feng Juang,et al.  An Interval Type-2 Neural Fuzzy Classifier Learned Through Soft Margin Minimization and its Human Posture Classification Application , 2015, IEEE Transactions on Fuzzy Systems.

[78]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[79]  Minvydas Ragulskis,et al.  Evolutionary algorithms for the selection of time lags for time series forecasting by fuzzy inference systems , 2010, Neurocomputing.

[80]  Sundaram Suresh,et al.  A meta-cognitive interval type-2 fuzzy inference system and its projection based learning algorithm , 2014, Evol. Syst..

[81]  J. Alberto Espinosa,et al.  Big Data: Issues and Challenges Moving Forward , 2013, 2013 46th Hawaii International Conference on System Sciences.

[82]  Paris A. Mastorocostas,et al.  A recurrent fuzzy-neural model for dynamic system identification , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[83]  Chao Liu,et al.  Short-term prediction of wind power using EMD and chaotic theory , 2012 .

[84]  Oscar Castillo,et al.  Generalized type-2 fuzzy weight adjustment for backpropagation neural networks in time series prediction , 2015, Inf. Sci..

[85]  Shih-Hsu Huang,et al.  VLSI implementation of type-2 fuzzy inference processor , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[86]  Oscar Castillo,et al.  Universal Approximation of a Class of Interval Type-2 Fuzzy Neural Networks in Nonlinear Identification , 2013, Adv. Fuzzy Syst..

[87]  F. Takens Detecting strange attractors in turbulence , 1981 .

[88]  J. Mendel Uncertain Rule-Based Fuzzy Logic Systems: Introduction and New Directions , 2001 .

[89]  C. Kasai,et al.  Real-Time Two-Dimensional Blood Flow Imaging Using an Autocorrelation Technique , 1985, IEEE 1985 Ultrasonics Symposium.

[90]  Sung-Kwun Oh,et al.  The Design of Optimized Type-2 Fuzzy Neural Networks and Its Application , 2009 .

[91]  Meie Shen,et al.  Optimal Selection of Parameters for Nonuniform Embedding of Chaotic Time Series Using Ant Colony Optimization , 2013, IEEE Transactions on Cybernetics.

[92]  Chia-Feng Juang,et al.  A Self-Evolving Interval Type-2 Fuzzy Neural Network With Online Structure and Parameter Learning , 2008, IEEE Transactions on Fuzzy Systems.

[93]  Min Han,et al.  Prediction of chaotic time series based on the recurrent predictor neural network , 2004, IEEE Transactions on Signal Processing.

[94]  Wang Yan-Chen,et al.  Prediction model of supply chain demand based on fuzzy neural network with chaotic time series , 2009, Proceedings of 2013 IEEE International Conference on Service Operations and Logistics, and Informatics.

[95]  Yang-Yin Lin,et al.  A Recurrent Self-Evolving Interval Type-2 Fuzzy Neural Network for Dynamic System Processing , 2009, IEEE Transactions on Fuzzy Systems.

[96]  Jerry M. Mendel,et al.  Type-2 fuzzy sets and systems: an overview , 2007, IEEE Computational Intelligence Magazine.

[97]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[98]  Tie Qiu,et al.  Multivariate Chaotic Time Series Prediction Based on Improved Grey Relational Analysis , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[99]  Min Han,et al.  Support Vector Echo-State Machine for Chaotic Time-Series Prediction , 2007, IEEE Transactions on Neural Networks.

[100]  Georg A. Gottwald,et al.  On the Implementation of the 0-1 Test for Chaos , 2009, SIAM J. Appl. Dyn. Syst..

[101]  Phayung Meesad,et al.  An optimal design for type-2 fuzzy logic system using hybrid of chaos firefly algorithm and genetic algorithm and its application to sea level prediction , 2014, J. Intell. Fuzzy Syst..

[102]  Yeong-Hwa Chang,et al.  Adaptive Dynamic Surface Control for Uncertain Nonlinear Systems With Interval Type-2 Fuzzy Neural Networks , 2014, IEEE Transactions on Cybernetics.

[103]  L. Cao Practical method for determining the minimum embedding dimension of a scalar time series , 1997 .

[104]  Oscar Castillo,et al.  A hybrid learning algorithm for a class of interval type-2 fuzzy neural networks , 2009, Inf. Sci..

[105]  Ching-Hung Lee,et al.  TYPE-2 FUZZY NEURAL NETWORK SYSTEMS AND LEARNING , 2002 .

[106]  Chia-Feng Juang,et al.  A Type-2 Self-Organizing Neural Fuzzy System and Its FPGA Implementation , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[107]  Jerry M. Mendel,et al.  Comments on “Interval Type-2 Fuzzy Sets are Generalization of Interval-Valued Fuzzy Sets: Towards a Wide View on Their Relationship” , 2015, IEEE Transactions on Fuzzy Systems.

[108]  Andrew J. Majda,et al.  Arctic Sea Ice Reemergence: The Role of Large-Scale Oceanic and Atmospheric Variability* , 2014 .

[109]  Oscar Castillo,et al.  Application of interval type-2 fuzzy neural networks in non-linear identification and time series prediction , 2013, Soft Computing.

[110]  Mahardhika Pratama,et al.  An Incremental Learning of Concept Drifts Using Evolving Type-2 Recurrent Fuzzy Neural Networks , 2017, IEEE Transactions on Fuzzy Systems.

[111]  Sreenatha G. Anavatti,et al.  Evolving type-2 web news mining , 2017, Appl. Soft Comput..

[112]  Veda C. Storey,et al.  Business Intelligence and Analytics: From Big Data to Big Impact , 2012, MIS Q..

[113]  Zhaohong Deng,et al.  Generalized Hidden-Mapping Ridge Regression, Knowledge-Leveraged Inductive Transfer Learning for Neural Networks, Fuzzy Systems and Kernel Methods , 2014, IEEE Transactions on Cybernetics.

[114]  Plamen P. Angelov,et al.  Parsimonious random vector functional link network for data streams , 2017, Inf. Sci..

[115]  Chin-Teng Lin,et al.  A Mutually Recurrent Interval Type-2 Neural Fuzzy System (MRIT2NFS) With Self-Evolving Structure and Parameters , 2013, IEEE Transactions on Fuzzy Systems.

[116]  J. Yorke,et al.  Period Three Implies Chaos , 1975 .

[117]  Francisco Herrera,et al.  Interval Type-2 Fuzzy Sets are Generalization of Interval-Valued Fuzzy Sets: Toward a Wider View on Their Relationship , 2015, IEEE Transactions on Fuzzy Systems.