Improving Generalization of Fuzzy IF--THEN Rules by Maximizing Fuzzy Entropy

When fuzzy IF-THEN rules initially extracted from data have not a satisfying performance, we consider that the rules require refinement. Distinct from most existing rule-refinement approaches that are based on the further reduction of training error, this paper proposes a new rule-refinement scheme that is based on the maximization of fuzzy entropy on the training set. The new scheme, which is realized by solving a quadratic programming problem, is expected to have the advantages of improving the generalization capability of initial fuzzy IF-THEN rules and simultaneously overcoming the overfitting of refinement. Experimental results on a number of selected databases demonstrate the expected improvement of generalization capability and the prevention of overfitting by a comparison of both training and testing accuracy before and after the refinement.

[1]  Hisao Ishibuchi,et al.  Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems , 1997, Fuzzy Sets Syst..

[2]  J.W.T. Lee,et al.  Refinement of generated fuzzy production rules by using a fuzzy neural network , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[3]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[4]  Shigeo Abe,et al.  A neural-network-based fuzzy classifier , 1995, IEEE Trans. Syst. Man Cybern..

[5]  Sushmita Mitra,et al.  Neuro-fuzzy rule generation: survey in soft computing framework , 2000, IEEE Trans. Neural Networks Learn. Syst..

[6]  Patrick Siarry,et al.  Fuzzy rule base learning through simulated annealing , 1999, Fuzzy Sets Syst..

[7]  János Abonyi,et al.  Learning fuzzy classification rules from labeled data , 2003, Inf. Sci..

[8]  Jianwei Zhang,et al.  Extracting compact fuzzy rules based on adaptive data approximation using B-splines , 2000, Inf. Sci..

[9]  Juan Luis Castro,et al.  Extraction of fuzzy rules from support vector machines , 2007, Fuzzy Sets Syst..

[10]  M. Shaw,et al.  Induction of fuzzy decision trees , 1995 .

[11]  Rudolf Kruse,et al.  A neuro-fuzzy method to learn fuzzy classification rules from data , 1997, Fuzzy Sets Syst..

[12]  M. Ait Kbir,et al.  Hierarchical fuzzy partition for pattern classification with fuzzy if-then rules , 2000, Pattern Recognit. Lett..

[13]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[14]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[15]  Sankar K. Pal.  Fuzzy set theoretic measures for automatic feature evaluation: II , 1992, Inf. Sci..

[16]  D. S. Yeung,et al.  Improving learning accuracy of fuzzy decision trees by hybrid neural networks , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[17]  Settimo Termini,et al.  A Definition of a Nonprobabilistic Entropy in the Setting of Fuzzy Sets Theory , 1972, Inf. Control..

[18]  H. D. Cheng,et al.  Thresholding using two-dimensional histogram and fuzzy entropy principle , 2000, IEEE Trans. Image Process..

[19]  T. Hong,et al.  Learning a coverage set of maximally general fuzzy rules by rough sets , 2000 .

[20]  Nikola K. Kasabov,et al.  Learning fuzzy rules and approximate reasoning in fuzzy neural networks and hybrid systems , 1996, Fuzzy Sets Syst..

[21]  Soung Hie Kim,et al.  A note on the fuzzy weighted additive rule , 1996, Fuzzy Sets Syst..

[22]  Plamen P. Angelov,et al.  Automatic generation of fuzzy rule-based models from data by genetic algorithms , 2003, Inf. Sci..

[23]  Hisao Ishibuchi,et al.  Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining , 2004, Fuzzy Sets Syst..

[24]  Xizhao Wang,et al.  Handling interaction in fuzzy production rule reasoning , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  R. Ramakrishnan,et al.  The fuzzy weighted additive rule , 1992 .

[26]  Hisao Ishibuchi,et al.  Rule weight specification in fuzzy rule-based classification systems , 2005, IEEE Transactions on Fuzzy Systems.

[27]  Hisao Ishibuchi,et al.  A simple but powerful heuristic method for generating fuzzy rules from numerical data , 1997, Fuzzy Sets Syst..

[28]  Eghbal G. Mansoori,et al.  Weighting fuzzy classification rules using receiver operating characteristics (ROC) analysis , 2007, Inf. Sci..

[29]  D. Dubois,et al.  Fundamentals of fuzzy sets , 2000 .

[30]  Hisao Ishibuchi,et al.  Effect of rule weights in fuzzy rule-based classification systems , 2001, IEEE Trans. Fuzzy Syst..

[31]  Shigeo Abe,et al.  A method for fuzzy rules extraction directly from numerical data and its application to pattern classification , 1995, IEEE Trans. Fuzzy Syst..

[32]  P. Lau,et al.  An enhanced weighted fuzzy reasoning algorithm for engineering applications , 1997 .

[33]  Ignacio Requena,et al.  A learning procedure to identify weighted rules by neural networks , 1995 .

[34]  Claudio Moraga,et al.  Extracting fuzzy if-then rules by using the information matrix technique , 2005, J. Comput. Syst. Sci..

[35]  E.C.C. Tsang,et al.  A comparative study on heuristic algorithms for generating fuzzy decision trees , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[36]  F. Klawonn,et al.  Evolving Fuzzy Rule-based Classifiers , 2007, 2007 IEEE Symposium on Computational Intelligence in Image and Signal Processing.

[37]  L. Kuncheva,et al.  Improving classification performance using fuzzy MLP and two-level selective partitioning of the feature space , 1995 .

[38]  Yadong Wang,et al.  A new approach to fuzzy rule generation: fuzzy extension matrix , 2001, Fuzzy Sets Syst..

[39]  Etienne E. Kerre,et al.  Fuzzy If-Then Rules in Computational Intelligence , 2000 .

[40]  H. Ishibuchi,et al.  Distributed representation of fuzzy rules and its application to pattern classification , 1992 .

[41]  Anil K. Jain,et al.  Dimensionality reduction using genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[42]  Magne Setnes,et al.  GA-fuzzy modeling and classification: complexity and performance , 2000, IEEE Trans. Fuzzy Syst..

[43]  Yi-Chung Hu,et al.  Finding fuzzy classification rules using data mining techniques , 2003, Pattern Recognit. Lett..

[44]  Wenbing Tao,et al.  Image segmentation by three-level thresholding based on maximum fuzzy entropy and genetic algorithm , 2003, Pattern Recognit. Lett..

[45]  Daniel S. Yeung,et al.  Weighted fuzzy production rules , 1997, Fuzzy Sets Syst..

[46]  Hisao Ishibuchi,et al.  Heuristic extraction of fuzzy classification rules using data mining techniques: an empirical study on benchmark data sets , 2004, 2004 IEEE International Conference on Fuzzy Systems (IEEE Cat. No.04CH37542).

[47]  Antonio González Muñoz,et al.  SLAVE: a genetic learning system based on an iterative approach , 1999, IEEE Trans. Fuzzy Syst..