Reducing the complexity in genetic learning of accurate regression TSK rule-based systems

In many real problems the regression models have to be accurate but, also, interpretable in order to provide qualitative understanding of the system. In this realm, the use of fuzzy rule base systems, particularly TSK, is widely extended. TSK rules combine the interpretability and expressiveness of rules with the ability of fuzzy logic for representing uncertainty, and the precision of the polynomials in the consequents. In this paper we present a new genetic fuzzy system to automatically learn accurate and simple linguistic TSK fuzzy rule bases that accurately model regression problems. In order to reduce the complexity of the learned models while keeping a high accuracy, we propose a Genetic Fuzzy System which consists of three stages: instance selection, multi-granularity fuzzy discretization of the input variables, and the evolutionary learning of the rule base using Elastic Net regularization. This proposal was validated using 28 real-world datasets and compared with three state of the art genetic fuzzy systems. Results show that our approach obtains the simplest models while achieving a similar accuracy to the best approximative models.

[1]  Sophia Ananiadou,et al.  Stochastic Gradient Descent Training for L1-regularized Log-linear Models with Cumulative Penalty , 2009, ACL.

[2]  Manuel Mucientes,et al.  An instance selection algorithm for regression and its application in variance reduction , 2013, 2013 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[3]  Rafael Alcalá,et al.  METSK-HDe: A multiobjective evolutionary algorithm to learn accurate TSK-fuzzy systems in high-dimensional and large-scale regression problems , 2014, Inf. Sci..

[4]  William H. Press,et al.  Numerical Recipes 3rd Edition: The Art of Scientific Computing , 2007 .

[5]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[6]  Francisco Herrera,et al.  A multi-objective evolutionary method for learning granularities based on fuzzy discretization to improve the accuracy-complexity trade-off of fuzzy rule-based classification systems: D-MOFARC algorithm , 2014, Appl. Soft Comput..

[7]  Jesús Alcalá-Fdez,et al.  Genetic learning of accurate and compact fuzzy rule based systems based on the 2-tuples linguistic representation , 2007, Int. J. Approx. Reason..

[8]  Francisco Herrera,et al.  A Fast and Scalable Multiobjective Genetic Fuzzy System for Linguistic Fuzzy Modeling in High-Dimensional Regression Problems , 2011, IEEE Transactions on Fuzzy Systems.

[9]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[10]  Antonio A. Márquez,et al.  An efficient adaptive fuzzy inference system for complex and high dimensional regression problems in linguistic fuzzy modelling , 2013, Knowl. Based Syst..

[11]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[12]  Francisco Herrera,et al.  Prototype Selection for Nearest Neighbor Classification: Taxonomy and Empirical Study , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Francisco Herrera,et al.  A 2-tuple fuzzy linguistic representation model for computing with words , 2000, IEEE Trans. Fuzzy Syst..

[14]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[15]  María José del Jesús,et al.  KEEL: a software tool to assess evolutionary algorithms for data mining problems , 2008, Soft Comput..

[16]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[17]  F. Gomide,et al.  Ten years of genetic fuzzy systems: current framework and new trends , 2001, Proceedings Joint 9th IFSA World Congress and 20th NAFIPS International Conference (Cat. No. 01TH8569).

[18]  Francisco Herrera,et al.  A Survey of Discretization Techniques: Taxonomy and Empirical Analysis in Supervised Learning , 2013, IEEE Transactions on Knowledge and Data Engineering.

[19]  Francisco Herrera,et al.  A Review of the Application of Multiobjective Evolutionary Fuzzy Systems: Current Status and Further Directions , 2013, IEEE Transactions on Fuzzy Systems.

[20]  Francisco Herrera,et al.  A study on the application of instance selection techniques in genetic fuzzy rule-based classification systems: Accuracy-complexity trade-off , 2013, Knowl. Based Syst..

[21]  Francisco Herrera,et al.  A taxonomy for the crossover operator for real‐coded genetic algorithms: An experimental study , 2003, Int. J. Intell. Syst..

[22]  Jerry M. Mendel,et al.  Generating fuzzy rules by learning from examples , 1992, IEEE Trans. Syst. Man Cybern..

[23]  Elena Marchiori,et al.  Class Conditional Nearest Neighbor for Large Margin Instance Selection , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Manuel Mucientes,et al.  STAC: A web platform for the comparison of algorithms using statistical tests , 2015, 2015 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[25]  Ron Kohavi,et al.  Supervised and Unsupervised Discretization of Continuous Features , 1995, ICML.

[26]  Hisao Ishibuchi,et al.  Performance evaluation of fuzzy partitions with different fuzzification grades , 2002, 2002 IEEE World Congress on Computational Intelligence. 2002 IEEE International Conference on Fuzzy Systems. FUZZ-IEEE'02. Proceedings (Cat. No.02CH37291).