Architectures for evolving fuzzy rule-based classifiers

In this paper the recently introduced evolving fuzzy classifier method called eClass is studied in respect to its architecture and evolution of the fuzzy rule-base. The proposed classifier has an open/evolving structure and can start 'from scratch', learning and adapting to the new data samples. Alternatively, if an initial fuzzy rule-based classifier, generated beforehand in off-line mode or provided by the operator, exists then eClass can evolve this initial classifier in on-line mode. In other words, the fuzzy rule base will evolve incorporating new rules, modifying and/or, possibly, removing some of the previously existing ones. Additionally, the parameters of both, the antecedent and the consequent parts are adapted. Note that eClass can start with an empty rule-base, which is a unique feature of this approach. The proposed approach is free from user-specified parameters and the mechanism of forming new rules is very robust. In this paper, four different modelling architectures are described and compared. The architectures are based on (i) unsupervised cluster partitions, eClassC; (ii) Sugeno fuzzy models with singleton consequents, eClassA; (iii) Takagi-Sugeno fuzzy models with linear consequent functions, eClassB; and (iv) a multi-model classification architecture, where separate TS regression models are combined to form an overall classification output of the system, eClassM. A thorough comparison of the results when applying each of these architectures and the results using previously existing classifiers has been made using an online interactive self-adaptive image classification framework.

[1]  János Abonyi,et al.  Learning Fuzzy Classification Rules from Data , 2001 .

[2]  Plamen P. Angelov,et al.  Data-driven evolving fuzzy systems using eTS and FLEXFIS: comparative analysis , 2008, Int. J. Gen. Syst..

[3]  P. Angelov,et al.  Evolving Fuzzy Systems from Data Streams in Real-Time , 2006, 2006 International Symposium on Evolving Fuzzy Systems.

[4]  Rong Jin,et al.  A smoothed boosting algorithm using probabilistic output codes , 2005, ICML '05.

[5]  Robert Tibshirani,et al.  Discriminant Adaptive Nearest Neighbor Classification , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Ulrich Bodenhofer,et al.  Incremental Learning of Fuzzy Basis Function Networks with a Modified Version of Vector Quantization , 2006 .

[7]  Plamen Angelov,et al.  Evolving Rule-Based Models: A Tool For Design Of Flexible Adaptive Systems , 2002 .

[8]  P. Villar,et al.  A multiobjective genetic algorithm for feature selection and granularity learning in fuzzy-rule based classification systems , 2001, Proceedings Joint 9th IFSA World Congress and 20th NAFIPS International Conference (Cat. No. 01TH8569).

[9]  Graham C. Goodwin,et al.  Adaptive filtering prediction and control , 1984 .

[10]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[11]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[12]  Edwin Lughofer,et al.  An approach to model-based fault detection in industrial measurement systems with application to engine test benches , 2006 .

[13]  Frank Klawonn,et al.  Evolving Extended Naive Bayes Classifiers , 2006, Sixth IEEE International Conference on Data Mining - Workshops (ICDMW'06).

[14]  Plamen P. Angelov,et al.  On-line identification of MIMO evolving Takagi- Sugeno fuzzy models , 2004, 2004 IEEE International Conference on Fuzzy Systems (IEEE Cat. No.04CH37542).

[15]  Ronald R. Yager,et al.  Learning of Fuzzy Rules by Mountain Clustering , 1992 .

[16]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery: An Overview , 1996, Advances in Knowledge Discovery and Data Mining.

[17]  D.P. Filev,et al.  An approach to online identification of Takagi-Sugeno fuzzy models , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[18]  Martin V. Butz,et al.  Rule-Based Evolutionary Online Learning Systems - A Principled Approach to LCS Analysis and Design , 2006, Studies in Fuzziness and Soft Computing.

[19]  G. Wittum,et al.  Adaptive filtering , 1997 .

[20]  Plamen P. Angelov,et al.  Evolving Single- And Multi-Model Fuzzy Classifiers with FLEXFIS-Class , 2007, 2007 IEEE International Fuzzy Systems Conference.

[21]  R. Yager,et al.  Approximate Clustering Via the Mountain Method , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[22]  Shaoning Pang,et al.  Incremental linear discriminant analysis for classification of data streams , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[23]  Johannes Gehrke,et al.  Querying and mining data streams: you only get one look a tutorial , 2002, SIGMOD '02.

[24]  Plamen Angelov,et al.  Identification of Evolving Rule-based Models. , 2002 .

[25]  Stephen L. Chiu,et al.  Fuzzy Model Identification Based on Cluster Estimation , 1994, J. Intell. Fuzzy Syst..

[26]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[27]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[28]  F. Klawonn,et al.  Evolving Fuzzy Rule-based Classifiers , 2007, 2007 IEEE Symposium on Computational Intelligence in Image and Signal Processing.

[29]  Andrew H. Jazwinski,et al.  Adaptive filtering , 1969, Autom..

[30]  Plamen P. Angelov,et al.  A fuzzy controller with evolving structure , 2004, Inf. Sci..

[31]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[32]  Bart Kosko,et al.  Fuzzy Systems as Universal Approximators , 1994, IEEE Trans. Computers.

[33]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[34]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[35]  Geoff Hulten,et al.  Catching up with the Data: Research Issues in Mining Data Streams , 2001, DMKD.

[36]  E. Tronci,et al.  1996 , 1997, Affair of the Heart.

[37]  Plamen P. Angelov,et al.  Identification of evolving fuzzy rule-based models , 2002, IEEE Trans. Fuzzy Syst..

[38]  Hans C. van Houwelingen,et al.  The Elements of Statistical Learning, Data Mining, Inference, and Prediction. Trevor Hastie, Robert Tibshirani and Jerome Friedman, Springer, New York, 2001. No. of pages: xvi+533. ISBN 0‐387‐95284‐5 , 2004 .

[39]  Edwin Lughofer,et al.  FLEXFIS: A Variant for Incremental Learning of Takagi-Sugeno Fuzzy Systems , 2005, The 14th IEEE International Conference on Fuzzy Systems, 2005. FUZZ '05..