Likelihood-fuzzy analysis: From data, through statistics, to interpretable fuzzy classifiers

Abstract Data-driven extraction of knowledge naturally takes advantage from the use of statistics, since statistical approaches enable to summarize information embedded in the dataset. On the other hand, fuzzy logic has gained increasing importance in data analytics, since it can enable to build a transparent and interpretable knowledge base. In this paper, a method named Likelihood-Fuzzy Analysis (LFA) for translating statistical information coming from labeled data into a fuzzy classification system is proposed. The characteristics of the method and of the resulting fuzzy classifiers are underlined. They range from the capacity of managing a dataset eventually comprising heterogeneous variables and missing data, to the high predictability, good confidence measure in terms of class probabilities, and interpretability of the fuzzy classification model, by means of semantically interpretable fuzzy partitions and if–then rules. The application of the method on a number of benchmark datasets is presented, showing high performances and semantic power, with respect to well-established methods, including fuzzy systems and non-fuzzy approaches.

[1]  Henri Prade,et al.  Fuzzy sets and probability: misunderstandings, bridges and gaps , 1993, [Proceedings 1993] Second IEEE International Conference on Fuzzy Systems.

[2]  José M. Alonso,et al.  Special issue on interpretable fuzzy systems , 2011, Inf. Sci..

[3]  Giuseppe De Pietro,et al.  Hybridization of possibility theory and supervised clustering to build DSSs for classification in medicine , 2012, 2012 12th International Conference on Hybrid Intelligent Systems (HIS).

[4]  Thierry Denoeux,et al.  Clustering and classification of fuzzy data using the fuzzy EM algorithm , 2016, Fuzzy Sets Syst..

[5]  Giuseppe De Pietro,et al.  Interpretability indexes for Fuzzy classification in cognitive systems , 2016, 2016 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[6]  M. Friedman The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .

[7]  David M. Dutton,et al.  A review of machine learning , 1997, The Knowledge Engineering Review.

[8]  David P. Pancho,et al.  Quest for Interpretability-Accuracy Trade-off Supported by Fingrams into the Fuzzy Modeling Tool GUAJE , 2013, Int. J. Comput. Intell. Syst..

[9]  K. Yamada Probability-possibility transformation based on evidence theory , 2001, Proceedings Joint 9th IFSA World Congress and 20th NAFIPS International Conference (Cat. No. 01TH8569).

[10]  Francisco Herrera,et al.  Ten years of genetic fuzzy systems: current framework and new trends , 2004, Fuzzy Sets Syst..

[11]  Xizhao Wang,et al.  On the optimization of fuzzy decision trees , 2000, Fuzzy Sets Syst..

[12]  Brigitte Charnomordic,et al.  Learning interpretable fuzzy inference systems with FisPro , 2011, Inf. Sci..

[13]  D. Dubois,et al.  A Semantics for Possibility Theory Based on Likelihoods , 1997 .

[14]  Corrado Mencar,et al.  Interpretability of Fuzzy Systems , 2013, WILF.

[15]  Sergio Guadarrama,et al.  Fuzzy representations need a careful design , 2010, Int. J. Gen. Syst..

[16]  William W. Cohen Fast Effective Rule Induction , 1995, ICML.

[17]  Francisco Herrera,et al.  A Review of the Application of Multiobjective Evolutionary Fuzzy Systems: Current Status and Further Directions , 2013, IEEE Transactions on Fuzzy Systems.

[18]  D. Dubois,et al.  Fuzzy sets and statistical data , 1986 .

[19]  Etienne E. Kerre,et al.  Defuzzification: criteria and classification , 1999, Fuzzy Sets Syst..

[20]  Didier Dubois,et al.  Possibility theory and statistical reasoning , 2006, Comput. Stat. Data Anal..

[21]  José M. Alonso,et al.  Interpretability of Fuzzy Systems: Current Research Trends and Prospects , 2015, Handbook of Computational Intelligence.

[22]  Didier Dubois,et al.  Probability-Possibility Transformations, Triangular Fuzzy Sets, and Probabilistic Inequalities , 2004, Reliab. Comput..

[23]  Oscar Cordón,et al.  International Journal of Approximate Reasoning a Historical Review of Evolutionary Learning Methods for Mamdani-type Fuzzy Rule-based Systems: Designing Interpretable Genetic Fuzzy Systems , 2022 .

[24]  Moshe Sipper,et al.  A fuzzy-genetic approach to breast cancer diagnosis , 1999, Artif. Intell. Medicine.

[25]  Serge Guillaume,et al.  Designing fuzzy inference systems from data: An interpretability-oriented review , 2001, IEEE Trans. Fuzzy Syst..

[26]  F. Herrera,et al.  Accuracy Improvements in Linguistic Fuzzy Modeling , 2003 .

[27]  Vladik Kreinovich,et al.  Entropy conserving probability transforms and the entailment principle , 2007, Fuzzy Sets Syst..

[28]  Christoph F. Eick,et al.  Supervised clustering - algorithms and benefits , 2004, 16th IEEE International Conference on Tools with Artificial Intelligence.

[29]  José M. Alonso,et al.  Looking for a good fuzzy system interpretability index: An experimental approach , 2009, Int. J. Approx. Reason..

[30]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[31]  J. Casillas Interpretability issues in fuzzy modeling , 2003 .

[32]  M. Shaw,et al.  Induction of fuzzy decision trees , 1995 .

[33]  Giuseppe De Pietro,et al.  Fuzzy partitioning for clinical DSSs using statistical information transformed into possibility-based knowledge , 2014, Knowl. Based Syst..

[34]  Richard Weber,et al.  Fuzzy-ID3: A class of methods for automatic knowledge acquisition , 1992 .

[35]  Francisco Herrera,et al.  Genetic fuzzy systems: taxonomy, current research trends and prospects , 2008, Evol. Intell..

[36]  Francisco Herrera,et al.  Interpretability of linguistic fuzzy rule-based systems: An overview of interpretability measures , 2011, Inf. Sci..

[37]  Jerry M. Mendel,et al.  Generating fuzzy rules by learning from examples , 1992, IEEE Trans. Syst. Man Cybern..

[38]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[39]  Francisco Herrera,et al.  Genetic Fuzzy Systems - Evolutionary Tuning and Learning of Fuzzy Knowledge Bases , 2002, Advances in Fuzzy Systems - Applications and Theory.

[40]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[41]  M. Rosenblatt Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[42]  Eyke Hüllermeier,et al.  Why Fuzzy Decision Trees are Good Rankers , 2009, IEEE Transactions on Fuzzy Systems.

[43]  W. Pedrycz Why triangular membership functions , 1994 .

[44]  Leszek Rutkowski,et al.  Flexible neuro-fuzzy systems , 2003, IEEE Trans. Neural Networks.

[45]  Giulianella Coletti,et al.  Possibilistic and probabilistic likelihood functions and their extensions: Common features and specific characteristics , 2014, Fuzzy Sets Syst..

[46]  E. Parzen On Estimation of a Probability Density Function and Mode , 1962 .

[47]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[48]  G. C. Tiao,et al.  Bayesian inference in statistical analysis , 1973 .

[49]  Michael G. Madden,et al.  A neural network approach to predicting stock exchange movements using external factors , 2005, Knowl. Based Syst..

[50]  Anna Maria Fanelli,et al.  Interpretability constraints for fuzzy information granulation , 2008, Inf. Sci..

[51]  José M. Alonso,et al.  A Survey of Fuzzy Systems Software: Taxonomy, Current Research Trends, and Prospects , 2016, IEEE Transactions on Fuzzy Systems.

[52]  Eyke Hüllermeier,et al.  FURIA: an algorithm for unordered fuzzy rule induction , 2009, Data Mining and Knowledge Discovery.

[53]  Luis Magdalena,et al.  Interpretability Improvements to Find the Balance Interpretability-Accuracy in Fuzzy Modeling: An Overview , 2003 .

[54]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[55]  Jesús Alcalá-Fdez,et al.  Hybrid learning models to get the interpretability–accuracy trade-off in fuzzy modeling , 2006, Soft Comput..

[56]  Witold Pedrycz,et al.  C-fuzzy decision trees , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[57]  Witold Pedrycz,et al.  Fuzzy equalization in the construction of fuzzy sets , 2001, Fuzzy Sets Syst..

[58]  Michio Sugeno,et al.  Industrial Applications of Fuzzy Control , 1985 .

[59]  José M. Alonso,et al.  Generating Understandable and Accurate Fuzzy Rule-Based Systems in a Java Environment , 2011, WILF.

[60]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[61]  O. J. Dunn Multiple Comparisons among Means , 1961 .

[62]  N. Singpurwalla,et al.  Membership Functions and Probability Measures of Fuzzy Sets , 2004 .

[63]  Giulianella Coletti,et al.  Conditional probability and fuzzy information , 2006, Comput. Stat. Data Anal..

[64]  Ferenc Szeifert,et al.  Supervised fuzzy clustering for the identification of fuzzy classifiers , 2003, Pattern Recognit. Lett..

[65]  George J. Klir,et al.  A MATHEMATICAL ANALYSIS OF INFORMATION-PRESERVING TRANSFORMATIONS BETWEEN PROBABILISTIC AND POSSIBILISTIC FORMULATIONS OF UNCERTAINTY , 1992 .

[66]  Giuseppe De Pietro,et al.  Transforming probability distributions into membership functions of fuzzy classes: A hypothesis test approach , 2013, Fuzzy Sets Syst..

[67]  L. Zadeh Fuzzy sets as a basis for a theory of possibility , 1999 .

[68]  Giuseppe De Pietro,et al.  An extensible six-step methodology to automatically generate fuzzy DSSs for diagnostic applications , 2013, BMC Bioinformatics.

[69]  Ralf Mikut,et al.  Interpretability issues in data-based learning of fuzzy systems , 2005, Fuzzy Sets Syst..

[70]  Jonathan Lawry,et al.  Decision tree learning with fuzzy labels , 2005, Inf. Sci..

[71]  Jonathan Lawry,et al.  A framework for linguistic modelling , 2004, Artif. Intell..

[72]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[73]  Sébastien Destercke,et al.  An extension of the FURIA classification algorithm to low quality data through fuzzy rankings and its application to the early diagnosis of dyslexia , 2016, Neurocomputing.

[74]  M. Friedman A Comparison of Alternative Tests of Significance for the Problem of $m$ Rankings , 1940 .

[75]  Irina Rish,et al.  An empirical study of the naive Bayes classifier , 2001 .

[76]  Krzysztof Cpalka,et al.  Design of Interpretable Fuzzy Systems , 2017, Studies in Computational Intelligence.

[77]  J. Ross Quinlan,et al.  Improved Use of Continuous Attributes in C4.5 , 1996, J. Artif. Intell. Res..

[78]  Rudolf Kruse,et al.  How the learning of rule weights affects the interpretability of fuzzy systems , 1998, 1998 IEEE International Conference on Fuzzy Systems Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36228).

[79]  Giuseppe De Pietro,et al.  An evolutionary-fuzzy DSS for assessing health status in multiple sclerosis disease , 2011, Int. J. Medical Informatics.

[80]  M. Narasimha Murty,et al.  Knowledge-based association rule mining using AND-OR taxonomies , 2003, Knowl. Based Syst..

[81]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[82]  Massimo Esposito,et al.  Insights into Interpretability of Neuro-Fuzzy Systems , 2015, IFSA-EUSFLAT.

[83]  Robert Babuška,et al.  An overview of fuzzy modeling for control , 1996 .

[84]  Ebrahim H. Mamdani,et al.  An Experiment in Linguistic Synthesis with a Fuzzy Logic Controller , 1999, Int. J. Hum. Comput. Stud..

[85]  Antonio A. Márquez,et al.  A multi-objective evolutionary algorithm with an interpretability improvement mechanism for linguistic fuzzy systems with adaptive defuzzification , 2010, International Conference on Fuzzy Systems.