HANDLING HIGHLY-DIMENSIONAL CLASSIFICATION TASKS WITH HIERARCHICAL GENETIC FUZZY RULE-BASED CLASSIFIERS

Many modern classification tasks are defined in highly-dimensional feature spaces. The derivation of high-performing genetic fuzzy rule-based classification systems (GFRBCSs) in such scenarios is a non-trivial task. This paper presents a framework for increasing the performance of GFRBCSs by creating a hierarchical fuzzy rule-based classifier. The proposed system is constructed through repeated invocations to a base GFRBCS procedure, considering at each step an input space fuzzy partition of a certain granularity. The best performing rules are inserted in the hierarchical rule base and the process is repeated again, considering a thicker granularity. The employed boosting scheme guides the algorithm in creating new rules to treat uncovered or misclassified patterns, thus monotonically increasing the performance of the classifier. Extensive experimental analysis in a number of real-world high-dimensional classification tasks proves the effectiveness of the proposed approach in increasing the performance of the base classifier, maintaining its interpretability to a considerable degree.

[1]  Kenneth DeJong Evolutionary computation: a unified approach , 2007, GECCO.

[2]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[3]  Francisco Herrera,et al.  Ten years of genetic fuzzy systems: current framework and new trends , 2004, Fuzzy Sets Syst..

[4]  Yoram Singer,et al.  A simple, fast, and effective rule learner , 1999, AAAI 1999.

[5]  Ioannis B. Theocharis,et al.  Reducing the Complexity of Genetic Fuzzy Classifiers in Highly-Dimensional Classification Problems , 2012, Int. J. Comput. Intell. Syst..

[6]  Q GanJohn,et al.  Low-level interpretability and high-level interpretability , 2008 .

[7]  William W. Cohen Fast Effective Rule Induction , 1995, ICML.

[8]  María José del Jesús,et al.  Hierarchical fuzzy rule based classification systems with genetic rule selection for imbalanced data-sets , 2009, Int. J. Approx. Reason..

[9]  F. Herrera,et al.  Analyzing the reasoning mechanisms in fuzzy rule based classification systems. , 1998 .

[10]  María José del Jesús,et al.  Induction of fuzzy-rule-based classifiers with evolutionary boosting algorithms , 2004, IEEE Transactions on Fuzzy Systems.

[11]  Francisco Herrera,et al.  Multiobjective genetic fuzzy rule selection of single granularity-based fuzzy classification rules and its interaction with the lateral tuning of membership functions , 2011, Soft Comput..

[12]  Hisao Ishibuchi,et al.  Hybridization of fuzzy GBML approaches for pattern classification problems , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[13]  María José del Jesús,et al.  Analysing the Hierarchical Fuzzy Rule Based Classification Systems with genetic rule selection , 2010, 2010 4th International Workshop on Genetic and Evolutionary Fuzzy Systems (GEFS).

[14]  Francisco Herrera,et al.  Genetic Fuzzy Systems - Evolutionary Tuning and Learning of Fuzzy Knowledge Bases , 2002, Advances in Fuzzy Systems - Applications and Theory.

[15]  Hisao Ishibuchi,et al.  Classification and modeling with linguistic information granules - advanced approaches to linguistic data mining , 2004, Advanced information processing.

[16]  Hisao Ishibuchi,et al.  Performance evaluation of fuzzy classifier systems for multidimensional pattern classification problems , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[17]  Francisco Herrera,et al.  Linguistic modeling by hierarchical systems of linguistic rules , 2002, IEEE Trans. Fuzzy Syst..

[18]  Ioannis B. Theocharis,et al.  A Genetic Fuzzy-Rule-Based Classifier for Land Cover Classification From Hyperspectral Imagery , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Liangpei Zhang,et al.  A pixel shape index coupled with spectral information for classification of high spatial resolution remotely sensed imagery , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Antonio J. Rivera,et al.  GP-COACH: Genetic Programming-based learning of COmpact and ACcurate fuzzy rule-based classification systems for High-dimensional problems , 2010, Inf. Sci..

[21]  Francisco Herrera,et al.  Interpretability of linguistic fuzzy rule-based systems: An overview of interpretability measures , 2011, Inf. Sci..

[22]  John Q. Gan,et al.  Low-level interpretability and high-level interpretability: a unified view of data-driven interpretable fuzzy system modelling , 2008, Fuzzy Sets Syst..

[23]  Antonio A. Márquez,et al.  A Mechanism to Improve the Interpretability of Linguistic Fuzzy Systems with Adaptive Defuzzification based on the use of a Multi-objective Evolutionary Algorithm , 2012, Int. J. Comput. Intell. Syst..

[24]  José M. Alonso,et al.  Special issue on interpretable fuzzy systems , 2011, Inf. Sci..

[25]  María José del Jesús,et al.  KEEL: a software tool to assess evolutionary algorithms for data mining problems , 2008, Soft Comput..

[26]  Dimitris G. Stavrakoudis,et al.  A Boosted Genetic Fuzzy Classifier for land cover classification of remote sensing imagery , 2011 .

[27]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[28]  Frank Hoffmann,et al.  Combining boosting and evolutionary algorithms for learning of fuzzy classification rules , 2004, Fuzzy Sets Syst..

[29]  Ioannis B. Theocharis,et al.  Decision Fusion of GA Self-Organizing Neuro-Fuzzy Multilayered Classifiers for Land Cover Classification Using Textural and Spectral Features , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[30]  Francisco Herrera,et al.  A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter Optimization , 2009, J. Heuristics.

[31]  Ioannis B. Theocharis,et al.  SVM-FuzCoC: A novel SVM-based feature selection method using a fuzzy complementary criterion , 2010, Pattern Recognit..

[32]  Ioannis B. Theocharis,et al.  Subject Recognition Based on Ground Reaction Force Measurements of Gait Signals , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[33]  Hisao Ishibuchi,et al.  Rule weight specification in fuzzy rule-based classification systems , 2005, IEEE Transactions on Fuzzy Systems.

[34]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[35]  Liangpei Zhang,et al.  Classification and Extraction of Spatial Features in Urban Areas Using High-Resolution Multispectral Imagery , 2007, IEEE Geoscience and Remote Sensing Letters.

[36]  José M. Alonso,et al.  HILK++: an interpretability-guided fuzzy modeling methodology for learning readable and comprehensible fuzzy rule-based classifiers , 2011, Soft Comput..

[37]  David A. Landgrebe,et al.  Signal Theory Methods in Multispectral Remote Sensing , 2003 .

[38]  Hisao Ishibuchi,et al.  Selecting fuzzy if-then rules for classification problems using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[39]  Hisao Ishibuchi,et al.  Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining , 2004, Fuzzy Sets Syst..

[40]  Antonio González Muñoz,et al.  Table Ii Tc Pattern Recognition Result for 120 Eir Satellite Image Cases Selection of Relevant Features in a Fuzzy Genetic Learning Algorithm , 2001 .

[41]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[42]  Thomas Bäck,et al.  Evolutionary Algorithms in Theory and Practice , 1996 .

[43]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[44]  Beatrice Lazzerini,et al.  Learning knowledge bases of multi-objective evolutionary fuzzy systems by simultaneously optimizing accuracy, complexity and partition integrity , 2011, Soft Comput..

[45]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[46]  Hisao Ishibuchi,et al.  Voting in fuzzy rule-based systems for pattern classification problems , 1999, Fuzzy Sets Syst..

[47]  Nikos Koutsias,et al.  SVM-Based Fuzzy Decision Trees for Classification of High Spatial Resolution Remote Sensing Images , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[48]  Luis Magdalena,et al.  HILK: A new methodology for designing highly interpretable linguistic knowledge bases using the fuzzy logic formalism , 2008 .

[49]  Hong Yan,et al.  Fuzzy Algorithms: With Applications to Image Processing and Pattern Recognition , 1996, Advances in Fuzzy Systems - Applications and Theory.

[50]  Antonio González Muñoz,et al.  SLAVE: a genetic learning system based on an iterative approach , 1999, IEEE Trans. Fuzzy Syst..

[51]  M. Lozano,et al.  MOGUL: A methodology to obtain genetic fuzzy rule‐based systems under the iterative rule learning approach , 1999 .

[52]  Francisco Herrera,et al.  Genetic fuzzy systems: taxonomy, current research trends and prospects , 2008, Evol. Intell..