A First Study on a Fuzzy Rule-Based Multiclassification System Framework Combining FURIA with Bagging and Feature Selection

In this work, we conduct a preliminary study considering a fuzzy rule-based multiclassification system design framework based on Fuzzy Unordered Rule Induction Algorithm (FURIA). This advanced method serves as the fuzzy classification rule learning algorithm to derive the component classifiers considering bagging combined with feature selection. We develop a study on the use of both bagging and feature selection to design a final FURIA-based fuzzy multiclassifier applied to ten popular UCI datasets. The results obtained show that this approach provides promising results.

[1]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[2]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[3]  Witold Pedrycz,et al.  Boosting of granular models , 2006, Fuzzy Sets Syst..

[4]  Oscar Cordón,et al.  Bi-criteria Genetic Selection of Bagging Fuzzy Rule-based Multiclassification Systems , 2009, IFSA/EUSFLAT Conf..

[5]  Piero P. Bonissone,et al.  A Fuzzy Random Forest: Fundamental for Design and Construction , 2008 .

[6]  Hisao Ishibuchi,et al.  Designing Fuzzy Ensemble Classifiers by Evolutionary Multiobjective Optimization with an Entropy-Based Diversity Criterion , 2006, 2006 Sixth International Conference on Hybrid Intelligent Systems (HIS'06).

[7]  William B. Yates,et al.  Engineering Multiversion Neural-Net Systems , 1996, Neural Computation.

[8]  Oscar Cordón,et al.  On the Use of Bagging, Mutual Information-Based Feature Selection and Multicriteria Genetic Algorithms to Design Fuzzy Rule-Based Classification Ensembles , 2008, 2008 Eighth International Conference on Hybrid Intelligent Systems.

[9]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[10]  Hisao Ishibuchi,et al.  Voting in fuzzy rule-based systems for pattern classification problems , 1999, Fuzzy Sets Syst..

[11]  Christophe Marsala,et al.  Data mining with ensembles of fuzzy decision trees , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[12]  Hisao Ishibuchi,et al.  Evolutionary multiobjective optimization for the design of fuzzy rule-based ensemble classifiers , 2006, Int. J. Hybrid Intell. Syst..

[13]  F. Herrera,et al.  A proposal on reasoning methods in fuzzy rule-based classification systems , 1999 .

[14]  C. N. Bouza,et al.  Handbook of Computational Statistics. Concepts and Methods , 2013 .

[15]  Oscar Cordón,et al.  A first study on bagging fuzzy rule-based classification systems with multicriteria genetic selection of the component classifiers , 2008, 2008 3rd International Workshop on Genetic and Evolving Systems.

[16]  Hiroyuki Honda,et al.  A New Reliable Cancer Diagnosis Method Using Boosted Fuzzy Classifier with a SWEEP Operator Method , 2005 .

[17]  Lawrence O. Hall,et al.  A Comparison of Decision Tree Ensemble Creation Techniques , 2007 .

[18]  Oscar Cordón,et al.  On the Combination of Accuracy and Diversity Measures for Genetic Selection of Bagging Fuzzy Rule-Based Multiclassification Systems , 2009, 2009 Ninth International Conference on Intelligent Systems Design and Applications.

[19]  María José del Jesús,et al.  Niching genetic feature selection algorithms applied to the design of fuzzy rule-based classification systems , 2007, 2007 IEEE International Fuzzy Systems Conference.

[20]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[21]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[22]  Frank Hoffmann,et al.  Combining boosting and evolutionary algorithms for learning of fuzzy classification rules , 2004, Fuzzy Sets Syst..

[23]  Yang Yu,et al.  Ensembling local learners ThroughMultimodal perturbation , 2005, IEEE Trans. Syst. Man Cybern. Part B.

[24]  Oscar Cordón,et al.  Comparing two genetic overproduce-and-choose strategies for fuzzy rule-based multiclassification systems generated by bagging and mutual information-based feature selection , 2010, Int. J. Hybrid Intell. Syst..

[25]  Saso Dzeroski,et al.  Combining Bagging and Random Subspaces to Create Better Ensembles , 2007, IDA.

[26]  Thomas G. Dietterich.  Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms , 1998, Neural Computation.

[27]  Piero P. Bonissone,et al.  A fuzzy random forest , 2010, Int. J. Approx. Reason..

[28]  Hisao Ishibuchi,et al.  Genetic rule selection with a multi-classifier coding scheme for ensemble classifier design , 2007, Int. J. Hybrid Intell. Syst..

[29]  María José del Jesús,et al.  Induction of fuzzy-rule-based classifiers with evolutionary boosting algorithms , 2004, IEEE Transactions on Fuzzy Systems.

[30]  Mauricio G. C. Resende,et al.  Greedy Randomized Adaptive Search Procedures , 1995, J. Glob. Optim..

[31]  Robert Sabourin,et al.  A dynamic overproduce-and-choose strategy for the selection of classifier ensembles , 2008, Pattern Recognit..

[32]  Lawrence O. Hall,et al.  Ensembles of Fuzzy Classifiers , 2007, 2007 IEEE International Fuzzy Systems Conference.

[33]  Luciano Sánchez,et al.  Boosting fuzzy rules in classification problems under single‐winner inference , 2007, Int. J. Intell. Syst..

[34]  Cezary Z. Janikow,et al.  Fuzzy decision trees: issues and methods , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[35]  Hiroyuki Honda,et al.  Lymphoma Prognostication from Expression Profiling Using a Combination Method of Boosting and Projective Adaptive Resonance Theory , 2006 .

[36]  Chen-An Tsai,et al.  Multi-class clustering and prediction in the analysis of microarray data. , 2005, Mathematical biosciences.

[37]  Witold Pedrycz,et al.  Fuzzy Adaptive Logic Networks as Hybrid Models of Quantitative Software Engineering , 2006, Intell. Autom. Soft Comput..

[38]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[39]  Hisao Ishibuchi,et al.  Classification and modeling with linguistic information granules - advanced approaches to linguistic data mining , 2004, Advanced information processing.

[40]  Eyke Hüllermeier,et al.  FURIA: an algorithm for unordered fuzzy rule induction , 2009, Data Mining and Knowledge Discovery.

[41]  Robert E. Schapire,et al.  The strength of weak learnability , 1990, Mach. Learn..

[42]  Eyke Hüllermeier,et al.  An Analysis of the FURIA Algorithm for Fuzzy Rule Induction , 2010, Advances in Machine Learning I.

[43]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[45]  Mykola Pechenizkiy,et al.  Diversity in search strategies for ensemble feature selection , 2005, Inf. Fusion.

[46]  D. Opitz,et al.  Popular Ensemble Methods: An Empirical Study , 1999, J. Artif. Intell. Res..

[47]  Rafal Scherer.  Boosting Ensemble of Relational Neuro-fuzzy Systems , 2006, ICAISC.

[48]  Michael Georgiopoulos,et al.  Boosted ARTMAP: Modifications to fuzzy ARTMAP motivated by boosting theory , 2006, Neural Networks.

[49]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[50]  Thomas G. Dietterich.  An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization , 2000, Machine Learning.