Multi-objective based Fuzzy Rule Based Systems (FRBSs) for trade-off improvement in accuracy and interpretability: A rule relevance point of view

Fuzzy rule-based systems (FRBSs) are a common alternative for applying fuzzy logic in different areas and real-world problems. The schemes and algorithms used to generate these types of systems imply that their performance can be analyzed from different points of view, not only model accuracy. Any model, including fuzzy models, needs to be sufficiently accurate, but other perspectives, such as interpretability, are also possible for the FRBSs. Thus, the Accuracy-Interpretability trade-off arises as a challenge for fuzzy systems, as approaches are currently able to generate FRBSs with different trade-offs.Here, rule Relevance is added to Accuracy and Interpretability for a better trade-off in FRBSs. These three factors are involved in this approach to make a rule selection using a multi-objective evolutionary algorithm.The proposal has been tested and compared with nine datasets, two linguistic and two scatter fuzzy algorithms, four measures of interpretability and two rule relevance formulations. The results have been analyzed for different views of Interpretability, Accuracy and Relevance, and the statistical tests have shown that significant improvements have been achieved. On the other hand, the Relevance-based role of fuzzy rules has been checked, and it has been shown that low Relevance rules have a relevant role for trade-off, while some rules with high Relevance must sometimes be removed to reach an adequate trade-off.

[1]  Laurence R. Horn,et al.  The handbook of pragmatics , 2004 .

[2]  Rafael Alcalá,et al.  Comparison and design of interpretable linguistic vs. scatter FRBSs: Gm3m generalization and new rule meaning index for global assessment and local pseudo-linguistic representation , 2014, Inf. Sci..

[3]  Oscar Cordón,et al.  International Journal of Approximate Reasoning a Historical Review of Evolutionary Learning Methods for Mamdani-type Fuzzy Rule-based Systems: Designing Interpretable Genetic Fuzzy Systems , 2022 .

[4]  John Q. Gan,et al.  Constructing L2-SVM-Based Fuzzy Classifiers in High-Dimensional Space With Automatic Model Selection and Fuzzy Rule Ranking , 2007, IEEE Transactions on Fuzzy Systems.

[5]  Yannis A. Dimitriadis,et al.  Learning from noisy information in FasArt and FasBack neuro-fuzzy systems , 2001, Neural Networks.

[6]  Luis Magdalena,et al.  Interpretability Improvements to Find the Balance Interpretability-Accuracy in Fuzzy Modeling: An Overview , 2003 .

[7]  Anna Maria Fanelli,et al.  Interpretability assessment of fuzzy knowledge bases: A cointension based approach , 2011, Int. J. Approx. Reason..

[8]  Paul Wing Hing Kwan,et al.  A knowledge-based Decision Support System for adaptive fingerprint identification that uses relevance feedback , 2015, Knowl. Based Syst..

[9]  M. J. Fuente,et al.  CHECKING ORTHOGONAL TRANSFORMATIONS AND GENETIC ALGORITHMS FOR SELECTION OF FUZZY RULES BASED ON INTERPRETABILITY-ACCURACY CONCEPTS , 2012 .

[10]  Rudolf Kruse,et al.  Neuro-fuzzy systems for function approximation , 1999, Fuzzy Sets Syst..

[11]  Luis Magdalena,et al.  Fuzzy Rule-Based Systems , 2015, Handbook of Computational Intelligence.

[12]  Malay Kumar Kundu,et al.  A graph-based relevance feedback mechanism in content-based image retrieval , 2015, Knowl. Based Syst..

[13]  Fakhreddine O. Karray,et al.  Soft Computing and Intelligent Systems Design, Theory, Tools and Applications , 2006, IEEE Transactions on Neural Networks.

[14]  Hiroshi Motoda,et al.  Feature Selection for Knowledge Discovery and Data Mining , 1998, The Springer International Series in Engineering and Computer Science.

[15]  José M. Alonso,et al.  Looking for a good fuzzy system interpretability index: An experimental approach , 2009, Int. J. Approx. Reason..

[16]  Uzay Kaymak,et al.  Similarity measures in fuzzy rule base simplification , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[17]  P. Salgado Rule generation for hierarchical collaborative fuzzy system , 2008 .

[18]  G. I. Sainz-Palmero,et al.  Complexity reduction and interpretability improvement for fuzzy rule systems based on simple interpretability measures and indices by bi-objective evolutionary rule selection , 2012, SOCO 2012.

[19]  Seiji Yamada,et al.  Learning filtering rulesets for ranking refinement in relevance feedback , 2005, Knowl. Based Syst..

[20]  Hisao Ishibuchi,et al.  Three-objective genetics-based machine learning for linguistic rule extraction , 2001, Inf. Sci..

[21]  J. Keynes A Treatise on Probability. , 1923 .

[22]  Jorge Casillas,et al.  Learning consistent, complete and compact sets of fuzzy rules in conjunctive normal form for regression problems , 2008, Soft Comput..

[23]  Marco Laumanns,et al.  SPEA2: Improving the Strength Pareto Evolutionary Algorithm For Multiobjective Optimization , 2002 .

[24]  Magne Setnes,et al.  Compact and transparent fuzzy models and classifiers through iterative complexity reduction , 2001, IEEE Trans. Fuzzy Syst..

[25]  Amit Konar,et al.  Computational Intelligence: Principles, Techniques and Applications , 2005 .

[26]  Hisao Ishibuchi,et al.  Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning , 2007, Int. J. Approx. Reason..

[27]  Francisco Herrera,et al.  A Multi-Objective Genetic Algorithm for Tuning and Rule Selection to Obtain Accurate and Compact Linguistic Fuzzy Rule-Based Systems , 2007, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[28]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[29]  Beatrice Lazzerini,et al.  Learning knowledge bases of multi-objective evolutionary fuzzy systems by simultaneously optimizing accuracy, complexity and partition integrity , 2011, Soft Comput..

[30]  John Yen,et al.  Simplifying fuzzy rule-based models using orthogonal transformation methods , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[31]  Sébastien Destercke,et al.  Building an interpretable fuzzy rule base from data using Orthogonal Least Squares - Application to a depollution problem , 2007, Fuzzy Sets Syst..

[32]  Francisco Herrera,et al.  A Review of the Application of Multiobjective Evolutionary Fuzzy Systems: Current Status and Further Directions , 2013, IEEE Transactions on Fuzzy Systems.

[33]  Brigitte Charnomordic,et al.  A new method for inducing a set of interpretable fuzzy partitions and fuzzy inference systems from data , 2003 .

[34]  Francisco Herrera,et al.  A study on the application of instance selection techniques in genetic fuzzy rule-based classification systems: Accuracy-complexity trade-off , 2013, Knowl. Based Syst..

[35]  Cat Ho Nguyen,et al.  A discussion on interpretability of linguistic rule based systems and its application to solve regression problems , 2015, Knowl. Based Syst..

[36]  Francisco Herrera,et al.  Genetic fuzzy systems: taxonomy, current research trends and prospects , 2008, Evol. Intell..

[37]  Sangchul Won,et al.  A new approach to fuzzy modeling of nonlinear dynamic systems with noise: relevance vector learning mechanism , 2006, IEEE Transactions on Fuzzy Systems.

[38]  Hisao Ishibuchi,et al.  Multiobjective Genetic Fuzzy Systems , 2009 .

[39]  Francisco Herrera,et al.  Interpretability of linguistic fuzzy rule-based systems: An overview of interpretability measures , 2011, Inf. Sci..

[40]  María José del Jesús,et al.  KEEL: a software tool to assess evolutionary algorithms for data mining problems , 2008, Soft Comput..

[41]  Robert Ivor John,et al.  On Constructing Parsimonious Type-2 Fuzzy Logic Systems via Influential Rule Selection , 2009, IEEE Transactions on Fuzzy Systems.

[42]  Hannu Koivisto,et al.  Developing a bioaerosol detector using hybrid genetic fuzzy systems , 2008, Eng. Appl. Artif. Intell..

[43]  Antonio A. Márquez,et al.  A Mechanism to Improve the Interpretability of Linguistic Fuzzy Systems with Adaptive Defuzzification based on the use of a Multi-objective Evolutionary Algorithm , 2012, Int. J. Comput. Intell. Syst..

[44]  José M. Alonso,et al.  A Survey of Fuzzy Systems Software: Taxonomy, Current Research Trends, and Prospects , 2016, IEEE Transactions on Fuzzy Systems.

[45]  Gene H. Golub,et al.  Matrix computations , 1983 .

[46]  Hisao Ishibuchi,et al.  Repeated double cross-validation for choosing a single solution in evolutionary multi-objective fuzzy classifier design , 2013, Knowl. Based Syst..

[47]  Jerry M. Mendel,et al.  Generating fuzzy rules by learning from examples , 1992, IEEE Trans. Syst. Man Cybern..

[48]  Francisco Herrera,et al.  A three-stage evolutionary process for learning descriptive and approximate fuzzy-logic-controller knowledge bases from examples , 1997, Int. J. Approx. Reason..

[49]  Larry J. Eshelman,et al.  The CHC Adaptive Search Algorithm: How to Have Safe Search When Engaging in Nontraditional Genetic Recombination , 1990, FOGA.

[50]  Fakhri Karray,et al.  Soft Computing and Tools of Intelligent Systems Design: Theory and Applications , 2004 .

[51]  Anna Maria Fanelli,et al.  Interpretability constraints for fuzzy information granulation , 2008, Inf. Sci..

[52]  Birger Hjørland The foundation of the concept of relevance , 2010 .

[53]  Francisco Herrera,et al.  Multiobjective genetic fuzzy rule selection of single granularity-based fuzzy classification rules and its interaction with the lateral tuning of membership functions , 2011, Soft Comput..

[54]  José M. Alonso,et al.  HILK++: an interpretability-guided fuzzy modeling methodology for learning readable and comprehensible fuzzy rule-based classifiers , 2011, Soft Comput..

[55]  José M. Alonso,et al.  Interpretability of Fuzzy Systems: Current Research Trends and Prospects , 2015, Handbook of Computational Intelligence.

[56]  Magne Setnes Simplification and reduction of fuzzy rules , 2003 .

[57]  Hisao Ishibuchi,et al.  Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems , 1997, Fuzzy Sets Syst..

[58]  María José del Jesús,et al.  Revisiting Evolutionary Fuzzy Systems: Taxonomy, applications, new trends and challenges , 2015, Knowl. Based Syst..

[59]  John Q. Gan,et al.  Low-level interpretability and high-level interpretability: a unified view of data-driven interpretable fuzzy system modelling , 2008, Fuzzy Sets Syst..

[60]  Witold Pedrycz,et al.  Springer Handbook of Computational Intelligence , 2015, Springer Handbook of Computational Intelligence.

[61]  Birger Hjørland The foundation of the concept of relevance , 2009 .

[62]  Francisco Herrera,et al.  Integration of an Index to Preserve the Semantic Interpretability in the Multiobjective Evolutionary Rule Selection and Tuning of Linguistic Fuzzy Systems , 2010, IEEE Transactions on Fuzzy Systems.

[63]  Ralf Mikut,et al.  Interpretability issues in data-based learning of fuzzy systems , 2005, Fuzzy Sets Syst..

[64]  Witold Pedrycz Expressing Relevance Interpretability and Accuracy of Rule-Based Systems , 2003 .

[65]  David P. Pancho,et al.  FINGRAMS: Visual Representations of Fuzzy Rule-Based Inference for Expert Analysis of Comprehensibility , 2013, IEEE Transactions on Fuzzy Systems.

[66]  Huan Liu,et al.  Efficient Feature Selection via Analysis of Relevance and Redundancy , 2004, J. Mach. Learn. Res..

[67]  José Valente de Oliveira,et al.  Semantic constraints for membership function optimization , 1999, IEEE Trans. Syst. Man Cybern. Part A.

[68]  Angelika Krone,et al.  Data-based fuzzy rule test for fuzzy modelling , 2001, Fuzzy Sets Syst..

[69]  Hisao Ishibuchi,et al.  Selecting fuzzy if-then rules for classification problems using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[70]  Francisco Herrera,et al.  Hybridizing genetic algorithms with sharing scheme and evolution strategies for designing approximate fuzzy rule-based systems , 2001, Fuzzy Sets Syst..