Design of interpretable fuzzy rule-based classifiers using spectral analysis with structure and parameters optimization

This paper presents a design method for fuzzy rule-based systems that performs data modeling consistently according to the symbolic relations expressed by the rules. The focus of the model is the interpretability of the rules and the model's accuracy, such that it can be used as tool for data understanding. The number of rules is defined by the eigenstructure analysis of the similarity matrix, which is computed from data. The rule induction algorithm runs a clustering algorithm on the dataset and associates one rule to each cluster. Each rule is selected among all possible combinations of one-dimensional fuzzy sets, as the one nearest to a cluster's center. The rules are weighted in order to improve the classifier performance and the weights are computed by a bounded quadratic optimization problem. The model complexity is minimized in a structure selection search, performed by a genetic algorithm that selects simultaneously the most representative subset of variables and also the number of fuzzy sets in the fuzzy partition of the selected variables. The resulting model is evaluated on a set of benchmark datasets for classification problems. The results show that the proposed approach produces accurate and yet compact fuzzy classifiers. The resulting model is also evaluated from an interpretability point of view, showing how the rule weights provide additional information to help data understanding and model exploitation.

[1]  Francesco Masulli,et al.  A survey of kernel and spectral methods for clustering , 2008, Pattern Recognit..

[2]  T. Mexia,et al.  Author ' s personal copy , 2009 .

[3]  María José del Jesús,et al.  Special Issue on Genetic Fuzzy Systems and the Interpretability-Accuracy Trade-off , 2007, Int. J. Approx. Reason..

[4]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[5]  Hisao Ishibuchi,et al.  Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning , 2007, Int. J. Approx. Reason..

[6]  Mansoor Zolghadri Jahromi,et al.  A proposed method for learning rule weights in fuzzy rule-based classification systems , 2008, Fuzzy Sets Syst..

[7]  Héctor Pomares,et al.  Improving the accuracy while preserving the interpretability of fuzzy function approximators by means of multi-objective evolutionary algorithms , 2007, Int. J. Approx. Reason..

[8]  Kok-Leong Ong,et al.  Enhancing the Effectiveness of Clustering with Spectra Analysis , 2007, IEEE Transactions on Knowledge and Data Engineering.

[9]  Thomas F. Coleman,et al.  A Reflective Newton Method for Minimizing a Quadratic Function Subject to Bounds on Some of the Variables , 1992, SIAM J. Optim..

[10]  J. Bezdek,et al.  VAT: a tool for visual assessment of (cluster) tendency , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[11]  Eyke Hüllermeier,et al.  Fuzzy methods in machine learning and data mining: Status and prospects , 2005, Fuzzy Sets Syst..

[12]  Francisco Herrera,et al.  Generating the knowledge base of a fuzzy rule-based system by the genetic learning of the data base , 2001, IEEE Trans. Fuzzy Syst..

[13]  Lotfi A. Zadeh,et al.  Outline of a New Approach to the Analysis of Complex Systems and Decision Processes , 1973, IEEE Trans. Syst. Man Cybern..

[14]  Shaogang Gong,et al.  Spectral clustering with eigenvector selection , 2008, Pattern Recognit..

[15]  Hisao Ishibuchi,et al.  Effect of rule weights in fuzzy rule-based classification systems , 2001, IEEE Trans. Fuzzy Syst..

[16]  Alexandre Evsukoff,et al.  Learning Fuzzy Rule Based Classifier with Rule Weights Optimization and Structure Selection by a Genetic Algorithm , 2007, 2007 IEEE International Fuzzy Systems Conference.

[17]  Jim Esch Computational Intelligence Methods For Rule-Based Data Understanding , 2004, Proc. IEEE.

[18]  Hisao Ishibuchi,et al.  Rule weight specification in fuzzy rule-based classification systems , 2005, IEEE Transactions on Fuzzy Systems.

[19]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[20]  Serge Guillaume,et al.  Designing fuzzy inference systems from data: An interpretability-oriented review , 2001, IEEE Trans. Fuzzy Syst..

[21]  U. Feige,et al.  Spectral Graph Theory , 2015 .

[22]  Hisao Ishibuchi,et al.  Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining , 2004, Fuzzy Sets Syst..

[23]  Yaochu Jin,et al.  Fuzzy modeling of high-dimensional systems: complexity reduction and interpretability improvement , 2000, IEEE Trans. Fuzzy Syst..

[24]  Francisco Herrera,et al.  Ten years of genetic fuzzy systems: current framework and new trends , 2004, Fuzzy Sets Syst..

[25]  María José del Jesús,et al.  A proposal on reasoning methods in fuzzy rule-based classification systems , 1999, Int. J. Approx. Reason..

[26]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[27]  James C. Bezdek,et al.  Visual cluster validity for prototype generator clustering models , 2003, Pattern Recognit. Lett..

[28]  Eghbal G. Mansoori,et al.  Weighting fuzzy classification rules using receiver operating characteristics (ROC) analysis , 2007, Inf. Sci..

[29]  Eyke Hüllermeier,et al.  A systematic approach to the assessment of fuzzy association rules , 2006, Data Mining and Knowledge Discovery.

[30]  Kim-Fung Man,et al.  Multi-objective hierarchical genetic algorithm for interpretable fuzzy rule-based knowledge extraction , 2005, Fuzzy Sets Syst..

[31]  Giovanna Castellano,et al.  A neuro-fuzzy network to generate human-understandable knowledge from data , 2002, Cognitive Systems Research.

[32]  Sylvie Galichet,et al.  Structure identification and parameter optimization for non-linear fuzzy modeling , 2002, Fuzzy Sets Syst..

[33]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[34]  Jung-Hsien Chiang,et al.  Support vector learning mechanism for fuzzy rule-based modeling: a new approach , 2004, IEEE Transactions on Fuzzy Systems.

[35]  Yixin Chen,et al.  Support vector learning for fuzzy rule-based classification systems , 2003, IEEE Trans. Fuzzy Syst..

[36]  Li-Xin Wang,et al.  The WM method completed: a flexible fuzzy system approach to data mining , 2003, IEEE Trans. Fuzzy Syst..

[37]  Lotfi A. Zadeh,et al.  Please Scroll down for Article International Journal of General Systems Fuzzy Sets and Systems* Fuzzy Sets and Systems* , 2022 .

[38]  Juan Luis Castro,et al.  Extraction of fuzzy rules from support vector machines , 2007, Fuzzy Sets Syst..

[39]  Sébastien Destercke,et al.  Building an interpretable fuzzy rule base from data using Orthogonal Least Squares - Application to a depollution problem , 2007, Fuzzy Sets Syst..

[40]  Ralf Mikut,et al.  Interpretability issues in data-based learning of fuzzy systems , 2005, Fuzzy Sets Syst..

[41]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.