Evolving fuzzy classifiers using different model architectures

In this paper we present two novel approaches for on-line evolving fuzzy classifiers, called eClass and FLEXFIS-Class. Both methods can be applied with different model architectures, including single model (SM) with class labels as consequents, classification hyper-planes as consequents, and multi-model (MM) architecture. Additionally, eClass can have a multi-input-multi-output (MIMO) architecture with multiple hyper-planes as consequents of each fuzzy rule. The difference between MM and MIMO architectures is that the former one applies one separate and independent fuzzy rule-based (FRB) classifier for each class and is using an indicator labelling scheme, while the latter one applies a single FRB where the rules are MIMO rather than MISO. Both, eClass and FLEXFIS-Class methods are designed to work on a per-sample basis and are thus one-pass, incremental. Additionally, their structure (FRB) is evolving rather than fixed. It adapts their parameters in antecedent and consequent parts with any newly loaded sample. A special emphasis is placed on advanced issues for improving accuracy and robustness, including a thorough comparison between global and local learning of consequent functions, a novel approach for detecting of and reacting on drifts in the data streams and an enhanced outlier treatment strategy. The methods and their extensions according to the advanced issues are evaluated on one benchmark problem of handwritten images recognition as well as on a real-life problem of image classification framework, where images should be classified into good and bad ones during an on-line and interactive production process.

[1]  D. Nauck,et al.  NEFCLASS-X — a Soft Computing Tool to Build Readable Fuzzy Classifiers , 1998 .

[2]  János Abonyi,et al.  Learning Fuzzy Classification Rules from Data , 2001 .

[3]  L X Wang,et al.  Fuzzy basis functions, universal approximation, and orthogonal least-squares learning , 1992, IEEE Trans. Neural Networks.

[4]  Karl Johan Åström,et al.  Adaptive Control , 1989, Embedded Digital Control with Microcontrollers.

[5]  Magne Setnes,et al.  GA-fuzzy modeling and classification: complexity and performance , 2000, IEEE Trans. Fuzzy Syst..

[6]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[7]  Rudolf Kruse,et al.  A neuro-fuzzy method to learn fuzzy classification rules from data , 1997, Fuzzy Sets Syst..

[8]  M. MendelJ.,et al.  Fuzzy basis functions , 1995 .

[9]  D.P. Filev,et al.  Novelty Detection Based Machine Health Prognostics , 2006, 2006 International Symposium on Evolving Fuzzy Systems.

[10]  Geoff Hulten,et al.  Catching up with the Data: Research Issues in Mining Data Streams , 2001, DMKD.

[11]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[12]  P. Angelov,et al.  Evolving Fuzzy Systems from Data Streams in Real-Time , 2006, 2006 International Symposium on Evolving Fuzzy Systems.

[13]  Eyke Hüllermeier,et al.  Improving the interpretability of data-driven evolving fuzzy systems , 2005, EUSFLAT Conf..

[14]  János Abonyi,et al.  Learning fuzzy classification rules from labeled data , 2003, Inf. Sci..

[15]  Plamen P. Angelov,et al.  On-line identification of MIMO evolving Takagi- Sugeno fuzzy models , 2004, 2004 IEEE International Conference on Fuzzy Systems (IEEE Cat. No.04CH37542).

[16]  Alexander J. Smola,et al.  Support Vector Method for Function Approximation, Regression Estimation and Signal Processing , 1996, NIPS.

[17]  Ronald R. Yager,et al.  Learning of Fuzzy Rules by Mountain Clustering , 1992 .

[18]  Plamen Angelov,et al.  On-line Identification of MIMO Evolving Takagi- , 2004 .

[19]  Alexey Tsymbal,et al.  The problem of concept drift: definitions and related work , 2004 .

[20]  D. Ruppert The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[21]  Nadine N. Tschichold-Gürman,et al.  Generation and improvement of fuzzy classifiers with incremental learning using fuzzy RuleNet , 1995, SAC '95.

[22]  Philip D. Wasserman,et al.  Advanced methods in neural computing , 1993, VNR computer library.

[23]  Thorsten Joachims,et al.  Detecting Concept Drift with Support Vector Machines , 2000, ICML.

[24]  E. Lughofer Process Safety Enhancements for Data-Driven Evolving Fuzzy Models , 2006, 2006 International Symposium on Evolving Fuzzy Systems.

[25]  N. Draper,et al.  Applied Regression Analysis. , 1967 .

[26]  D.P. Filev,et al.  An approach to online identification of Takagi-Sugeno fuzzy models , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[27]  Stephen L. Chiu,et al.  Fuzzy Model Identification Based on Cluster Estimation , 1994, J. Intell. Fuzzy Syst..

[28]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[29]  Edwin Lughofer,et al.  FLEXFIS: A Robust Incremental Learning Approach for Evolving Takagi–Sugeno Fuzzy Models , 2008, IEEE Transactions on Fuzzy Systems.

[30]  F. Klawonn,et al.  Evolving Fuzzy Rule-based Classifiers , 2007, 2007 IEEE Symposium on Computational Intelligence in Image and Signal Processing.

[31]  Hazem Tawfik,et al.  Handoff algorithms based on fuzzy classifiers , 2000, IEEE Trans. Veh. Technol..

[32]  Gerhard Widmer,et al.  Learning in the presence of concept drift and hidden contexts , 2004, Machine Learning.

[33]  Rafael Santos,et al.  Creating fuzzy rules for image classification using biased data clustering , 1999, Electronic Imaging.

[34]  Plamen P. Angelov,et al.  An approach for fuzzy rule-base adaptation using on-line clustering , 2004, Int. J. Approx. Reason..

[35]  Hisao Ishibuchi,et al.  Performance evaluation of fuzzy classifier systems for multidimensional pattern classification problems , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[36]  Weihua Li,et al.  Recursive PCA for Adaptive Process Monitoring , 1999 .

[37]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[38]  Shaoning Pang,et al.  Incremental linear discriminant analysis for classification of data streams , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[39]  Shigeo Abe,et al.  A method for fuzzy rules extraction directly from numerical data and its application to pattern classification , 1995, IEEE Trans. Fuzzy Syst..

[40]  E. Backer,et al.  Fuzzy Incremental Learning of Expert Rules for a Knowledge-Based Anesthesia Monitor , 1996 .

[41]  Gert Cauwenberghs,et al.  SVM incremental learning, adaptation and optimization , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[42]  Plamen Angelov,et al.  Evolving Rule-Based Models: A Tool For Design Of Flexible Adaptive Systems , 2002 .

[43]  Nikola K. Kasabov,et al.  Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[44]  Dimitar Filev,et al.  Generation of Fuzzy Rules by Mountain Clustering , 1994, J. Intell. Fuzzy Syst..

[45]  O. Nasraoui,et al.  Complete expression trees for evolving fuzzy classifier systems with genetic algorithms and application to network intrusion detection , 2002, 2002 Annual Meeting of the North American Fuzzy Information Processing Society Proceedings. NAFIPS-FLINT 2002 (Cat. No. 02TH8622).

[46]  Qiang Shen,et al.  From approximative to descriptive fuzzy classifiers , 2002, IEEE Trans. Fuzzy Syst..

[47]  John Yen,et al.  Improving the interpretability of TSK fuzzy models by combining global learning and local learning , 1998, IEEE Trans. Fuzzy Syst..

[48]  Plamen P. Angelov,et al.  Simpl_eTS: a simplified method for learning evolving Takagi-Sugeno fuzzy models , 2005, The 14th IEEE International Conference on Fuzzy Systems, 2005. FUZZ '05..

[49]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[50]  Edwin Lughofer,et al.  Extensions of vector quantization for incremental clustering , 2008, Pattern Recognit..

[51]  David G. Stork,et al.  Pattern Classification , 1973 .