An Interval Type-2 Neural Fuzzy System for Online System Identification and Feature Elimination

We propose an integrated mechanism for discarding derogatory features and extraction of fuzzy rules based on an interval type-2 neural fuzzy system (NFS)-in fact, it is a more general scheme that can discard bad features, irrelevant antecedent clauses, and even irrelevant rules. High-dimensional input variable and a large number of rules not only enhance the computational complexity of NFSs but also reduce their interpretability. Therefore, a mechanism for simultaneous extraction of fuzzy rules and reducing the impact of (or eliminating) the inferior features is necessary. The proposed approach, namely an interval type-2 Neural Fuzzy System for online System Identification and Feature Elimination (IT2NFS-SIFE), uses type-2 fuzzy sets to model uncertainties associated with information and data in designing the knowledge base. The consequent part of the IT2NFS-SIFE is of Takagi-Sugeno-Kang type with interval weights. The IT2NFS-SIFE possesses a self-evolving property that can automatically generate fuzzy rules. The poor features can be discarded through the concept of a membership modulator. The antecedent and modulator weights are learned using a gradient descent algorithm. The consequent part weights are tuned via the rule-ordered Kalman filter algorithm to enhance learning effectiveness. Simulation results show that IT2NFS-SIFE not only simplifies the system architecture by eliminating derogatory/irrelevant antecedent clauses, rules, and features but also maintains excellent performance.

[1]  Nikhil R. Pal,et al.  Selecting Useful Groups of Features in a Connectionist Framework , 2008, IEEE Transactions on Neural Networks.

[2]  Yu-Ching Lin,et al.  Systems identification using type-2 fuzzy neural network (type-2 FNN) systems , 2003, Proceedings 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation. Computational Intelligence in Robotics and Automation for the New Millennium (Cat. No.03EX694).

[3]  Michael R. Berthold,et al.  Input features' impact on fuzzy decision processes , 2000, IEEE Trans. Syst. Man Cybern. Part B.

[4]  Chia-Feng Juang,et al.  A Self-Evolving Interval Type-2 Fuzzy Neural Network With Online Structure and Parameter Learning , 2008, IEEE Transactions on Fuzzy Systems.

[5]  Yüksel Özbay,et al.  Integration of type-2 fuzzy clustering and wavelet transform in a neural network based ECG classifier , 2011, Expert Syst. Appl..

[6]  Jerry M. Mendel,et al.  Equalization of nonlinear time-varying channels using type-2 fuzzy adaptive filters , 2000, IEEE Trans. Fuzzy Syst..

[7]  Hani Hagras,et al.  A hierarchical type-2 fuzzy logic control architecture for autonomous mobile robots , 2004, IEEE Transactions on Fuzzy Systems.

[8]  Chih-Ming Chen,et al.  An efficient fuzzy classifier with feature selection based on fuzzy entropy , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[9]  Gerardo M. Mendez,et al.  Hybrid learning mechanism for interval A2-C1 type-2 non-singleton type-2 Takagi-Sugeno-Kang fuzzy logic systems , 2013, Inf. Sci..

[10]  Inés Couso,et al.  Mutual information-based feature selection and partition design in fuzzy rule-based classifiers from vague data , 2008, Int. J. Approx. Reason..

[11]  Chin-Teng Lin,et al.  A TSK-Type-Based Self-Evolving Compensatory Interval Type-2 Fuzzy Neural Network (TSCIT2FNN) and Its Applications , 2014, IEEE Transactions on Industrial Electronics.

[12]  Meng Joo Er,et al.  A fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks , 2009, Neurocomputing.

[13]  Yuan Lan,et al.  Ensemble of online sequential extreme learning machine , 2009, Neurocomputing.

[14]  Nikhil R. Pal,et al.  Simultaneous Structure Identification and Fuzzy Rule Generation for Takagi–Sugeno Models , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Chia-Feng Juang,et al.  Hierarchical Cluster-Based Multispecies Particle-Swarm Optimization for Fuzzy-System Optimization , 2010, IEEE Transactions on Fuzzy Systems.

[16]  Nikhil R. Pal,et al.  A neuro-fuzzy scheme for simultaneous feature selection and fuzzy rule-based classification , 2004, IEEE Transactions on Neural Networks.

[17]  Oscar Castillo,et al.  Universal Approximation of a Class of Interval Type-2 Fuzzy Neural Networks Illustrated with the Case of Non-linear Identification , 2009, IFSA/EUSFLAT Conf..

[18]  Junfei Qiao,et al.  A Self-Organizing Fuzzy Neural Network Based on a Growing-and-Pruning Algorithm , 2010, IEEE Transactions on Fuzzy Systems.

[19]  Tzuu-Hseng S. Li,et al.  Construction of a neuron-fuzzy classification model based on feature-extraction approach , 2011, Expert Syst. Appl..

[20]  Hisao Ishibuchi,et al.  Selecting fuzzy if-then rules for classification problems using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[21]  Nikhil R. Pal,et al.  An Integrated Mechanism for Feature Selection and Fuzzy Rule Extraction for Classification , 2012, IEEE Transactions on Fuzzy Systems.

[22]  Chin-Teng Lin,et al.  A Mutually Recurrent Interval Type-2 Neural Fuzzy System (MRIT2NFS) With Self-Evolving Structure and Parameters , 2013, IEEE Transactions on Fuzzy Systems.

[23]  Oscar Castillo,et al.  Comparison of Hybrid Intelligent Systems, Neural Networks and Interval Type-2 Fuzzy Logic for Time Series Prediction , 2007, 2007 International Joint Conference on Neural Networks.

[24]  J. Mendel.  Uncertain Rule-Based Fuzzy Logic Systems: Introduction and New Directions , 2001 .

[25]  Hisao Ishibuchi,et al.  Performance evaluation of fuzzy classifier systems for multidimensional pattern classification problems , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[26]  Jerry M. Mendel,et al.  Applications of Type-2 Fuzzy Logic Systems to Forecasting of Time-series , 1999, Inf. Sci..

[27]  Jerry Mendel,et al.  Type-2 Fuzzy Sets and Systems: An Overview [corrected reprint] , 2007, IEEE Computational Intelligence Magazine.

[28]  R. John,et al.  Type-2 Fuzzy Logic: A Historical View , 2007, IEEE Computational Intelligence Magazine.

[29]  Rafael Colás,et al.  Modelling and control of coiling entry temperature using interval type-2 fuzzy logic systems , 2010 .

[30]  Lei Wang,et al.  Global and Local Structure Preservation for Feature Selection , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Jerry M. Mendel,et al.  Interval type-2 fuzzy logic systems , 2000, Ninth IEEE International Conference on Fuzzy Systems. FUZZ- IEEE 2000 (Cat. No.00CH37063).

[32]  Jerry M. Mendel,et al.  Computing derivatives in interval type-2 fuzzy logic systems , 2004, IEEE Transactions on Fuzzy Systems.

[33]  Michio Sugeno,et al.  A fuzzy-logic-based approach to qualitative modeling , 1993, IEEE Trans. Fuzzy Syst..

[34]  Jerry M. Mendel,et al.  Type-2 fuzzy logic systems , 1999, IEEE Trans. Fuzzy Syst..

[35]  Francisco Herrera,et al.  Learning the membership function contexts for mining fuzzy association rules by using genetic algorithms , 2009, Fuzzy Sets Syst..

[36]  Oscar Castillo,et al.  A hybrid learning algorithm for a class of interval type-2 fuzzy neural networks , 2009, Inf. Sci..

[37]  Ching-Hung Lee,et al.  TYPE-2 FUZZY NEURAL NETWORK SYSTEMS AND LEARNING , 2002 .

[38]  Yang-Yin Lin,et al.  A Recurrent Self-Evolving Interval Type-2 Fuzzy Neural Network for Dynamic System Processing , 2009, IEEE Transactions on Fuzzy Systems.

[39]  Cuntai Guan,et al.  eT2FIS: An Evolving Type-2 Neural Fuzzy Inference System , 2013, Inf. Sci..

[40]  Chia-Feng Juang,et al.  An Interval Type-2 Fuzzy-Neural Network With Support-Vector Regression for Noisy Regression Problems , 2010, IEEE Transactions on Fuzzy Systems.

[41]  Jerry M. Mendel,et al.  Type-2 fuzzy sets made simple , 2002, IEEE Trans. Fuzzy Syst..

[42]  Witold Pedrycz,et al.  Improving RBF networks performance in regression tasks by means of a supervised fuzzy clustering , 2006, Neurocomputing.

[43]  Bo Zhang,et al.  Automatic Fuzzy Rules Generation Using Fuzzy Genetic Algorithm , 2009, 2009 Sixth International Conference on Fuzzy Systems and Knowledge Discovery.

[44]  Juan R. Castro,et al.  Intelligent control using an Interval Type-2 Fuzzy Neural Network with a hybrid learning algorithm , 2008, 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence).

[45]  Chin-Teng Lin,et al.  An online self-constructing neural fuzzy inference network and its applications , 1998, IEEE Trans. Fuzzy Syst..

[46]  Faa-Jeng Lin,et al.  Adaptive Control of Two-Axis Motion Control System Using Interval Type-2 Fuzzy Neural Network , 2009, IEEE Transactions on Industrial Electronics.

[47]  Igor Skrjanc,et al.  Editorial A Successful Change From TNN to TNNLS and a Very Successful Year , 2013, IEEE Trans. Neural Networks Learn. Syst..

[48]  Ismael Lopez-Juarez,et al.  Finishing mill thread speed set-up and control by interval type 1 non-singleton type 2 fuzzy logic systems , 2012 .

[49]  Yinghua Lin,et al.  A new approach to fuzzy-neural system modeling , 1995, IEEE Trans. Fuzzy Syst..

[50]  T. Martin McGinnity,et al.  Design for Self-Organizing Fuzzy Neural Networks Based on Genetic Algorithms , 2006, IEEE Transactions on Fuzzy Systems.

[51]  Cuntai Guan,et al.  An Evolving Type-2 Neural Fuzzy Inference System , 2010, PRICAI.

[52]  H. B. Mitchell.  Pattern recognition using type-II fuzzy sets , 2005, Inf. Sci..

[53]  Hani Hagras.  Comments on "Dynamical Optimal Training for Interval Type-2 Fuzzy Neural Network (T2FNN) , 2006, IEEE Trans. Syst. Man Cybern. Part B.

[54]  Nikhil R. Pal,et al.  Integrated feature analysis and fuzzy rule-based system identification in a neuro-fuzzy paradigm , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[55]  Jia Zeng,et al.  Type-2 fuzzy hidden Markov models and their application to speech recognition , 2006, IEEE Transactions on Fuzzy Systems.

[56]  Chi-Hsu Wang,et al.  Dynamical optimal training for interval type-2 fuzzy neural network (T2FNN) , 2003, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[57]  Kazuyuki Murase,et al.  A new wrapper feature selection approach using neural network , 2010, Neurocomputing.

[58]  Russell C. Eberhart,et al.  Implementation of evolutionary fuzzy systems , 1999, IEEE Trans. Fuzzy Syst..

[59]  Jerry M. Mendel,et al.  Advances in type-2 fuzzy sets and systems , 2007, Inf. Sci..

[60]  Hisao Ishibuchi,et al.  Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems , 1997, Fuzzy Sets Syst..

[61]  Okyay Kaynak,et al.  Type 2 Fuzzy Neural Structure for Identification and Control of Time-Varying Plants , 2010, IEEE Transactions on Industrial Electronics.

[62]  Gin-Der Wu,et al.  A Maximizing-Discriminability-Based Self-Organizing Fuzzy Network for Classification Problems , 2010, IEEE Transactions on Fuzzy Systems.

[63]  Hwa Jen Yap,et al.  Application of genetic algorithm for fuzzy rules optimization on semi expert judgment automation using Pittsburg approach , 2012, Appl. Soft Comput..

[64]  T. Martin McGinnity,et al.  An approach for on-line extraction of fuzzy rules using a self-organising fuzzy neural network , 2005, Fuzzy Sets Syst..