A QUADRATIC MARGIN-BASED MODEL FOR WEIGHTING FUZZY CLASSIFICATION RULES INSPIRED BY SUPPORT VECTOR MACHINES

Recently, tuning the weights of the rules in Fuzzy Rule-Base Clas- sification Systems is researched in order to improve the accuracy of classifica- tion. In this paper, a margin-based optimization model, inspired by Support Vector Machine classifiers, is proposed to compute these fuzzy rule weights. This approach not only considers both accuracy and generalization criteria in a single objective function, but also is independent of any order in presenting data patterns or fuzzy rules. It has a global optimum solution and needs only one regularization parameter C to be adjusted. In addition, a rule reduction method is proposed to eliminating low weighted rules and having a compact rule-base. This method is compared with some greedy, reinforcement and lo- cal search rule weighting methods on 13 standard datasets. The experimental results show that, the proposed method significantly outperforms the other ones especially from the viewpoint of generalization.

[1]  H. Ishibuchi,et al.  Distributed representation of fuzzy rules and its application to pattern classification , 1992 .

[2]  A. de Sam Lazaro,et al.  A fine tuning method for fuzzy logic rule bases , 1994 .

[3]  LiMin Fu,et al.  Rule Generation from Neural Networks , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[4]  Hisao Ishibuchi,et al.  Selecting fuzzy if-then rules for classification problems using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[5]  Hisao Ishibuchi,et al.  Adaptive fuzzy rule-based classification systems , 1996, IEEE Trans. Fuzzy Syst..

[6]  Hong Yan,et al.  Fuzzy Algorithms: With Applications to Image Processing and Pattern Recognition , 1996, Advances in Fuzzy Systems - Applications and Theory.

[7]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[8]  Hisao Ishibuchi,et al.  Simple fuzzy rule-based classification systems perform well on commonly used real-world data sets , 1997, 1997 Annual Meeting of the North American Fuzzy Information Processing Society - NAFIPS (Cat. No.97TH8297).

[9]  Hisao Ishibuchi,et al.  Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems , 1997, Fuzzy Sets Syst..

[10]  Hisao Ishibuchi,et al.  Techniques and Applications of Neural Networks for Fuzzy Rule Approximation , 1999 .

[11]  María José del Jesús,et al.  A proposal on reasoning methods in fuzzy rule-based classification systems , 1999, Int. J. Approx. Reason..

[12]  Jason Weston,et al.  Support vector machines for multi-class pattern recognition , 1999, ESANN.

[13]  F. Herrera,et al.  A proposal on reasoning methods in fuzzy rule-based classification systems , 1999 .

[14]  Hisao Ishibuchi,et al.  Voting in fuzzy rule-based systems for pattern classification problems , 1999, Fuzzy Sets Syst..

[15]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[16]  Hisao Ishibuchi,et al.  Effect of rule weights in fuzzy rule-based classification systems , 2001, IEEE Trans. Fuzzy Syst..

[17]  F. Gomide,et al.  Ten years of genetic fuzzy systems: current framework and new trends , 2001, Proceedings Joint 9th IFSA World Congress and 20th NAFIPS International Conference (Cat. No. 01TH8569).

[18]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[19]  Shyi-Ming Chen,et al.  Generating weighted fuzzy rules from relational database systems for estimating values using genetic algorithms , 2003, IEEE Trans. Fuzzy Syst..

[20]  Francisco Herrera,et al.  Ten years of genetic fuzzy systems: current framework and new trends , 2004, Fuzzy Sets Syst..

[21]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[22]  Hisao Ishibuchi,et al.  Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining , 2004, Fuzzy Sets Syst..

[23]  George Forman,et al.  Learning from Little: Comparison of Classifiers Given Little Training , 2004, PKDD.

[24]  Hisao Ishibuchi,et al.  Classification and modeling with linguistic information granules - advanced approaches to linguistic data mining , 2004, Advanced information processing.

[25]  J. Langford.  Tutorial on Practical Prediction Theory for Classification , 2005, J. Mach. Learn. Res..

[26]  Ralf Mikut,et al.  Interpretability issues in data-based learning of fuzzy systems , 2005, Fuzzy Sets Syst..

[27]  Hisao Ishibuchi,et al.  Rule weight specification in fuzzy rule-based classification systems , 2005, IEEE Transactions on Fuzzy Systems.

[28]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[29]  Hisao Ishibuchi,et al.  A weighted fuzzy classifier and its application to image processing tasks , 2007, Fuzzy Sets Syst..

[30]  Eghbal G. Mansoori,et al.  USING DISTRIBUTION OF DATA TO ENHANCE PERFORMANCE OF FUZZY CLASSIFICATION SYSTEMS , 2007 .

[31]  Hisao Ishibuchi,et al.  Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning , 2007, Int. J. Approx. Reason..

[32]  Eghbal G. Mansoori,et al.  Weighting fuzzy classification rules using receiver operating characteristics (ROC) analysis , 2007, Inf. Sci..

[33]  S. García,et al.  An Extension on "Statistical Comparisons of Classifiers over Multiple Data Sets" for all Pairwise Comparisons , 2008 .

[34]  Mansoor Zolghadri Jahromi,et al.  A proposed method for learning rule weights in fuzzy rule-based classification systems , 2008, Fuzzy Sets Syst..

[35]  Reza Boostani,et al.  GENERATING FUZZY RULES FOR PROTEIN CLASSIFICATION , 2008 .

[36]  Jian Xiao,et al.  Trade-off between accuracy and interpretability: Experience-oriented fuzzy modeling via reduced-set vectors , 2009, Comput. Math. Appl..

[37]  Shyi-Ming Chen,et al.  A new weighted fuzzy rule interpolation method based on GA-based weights-learning techniques , 2010, 2010 International Conference on Machine Learning and Cybernetics.

[38]  Shyi-Ming Chen,et al.  Weighted Fuzzy Rule Interpolation Based on GA-Based Weight-Learning Techniques , 2011, IEEE Transactions on Fuzzy Systems.

[39]  Qinghua Hu,et al.  Large-margin nearest neighbor classifiers via sample weight learning , 2011, Neurocomputing.