An approximation to solve regression problems with a genetic fuzzy rule ordinal algorithm

Abstract Regression problems try estimating a continuous variable from a number of characteristics or predictors. Several proposals have been made for regression models based on the use of fuzzy rules; however, all these proposals make use of rule models in which the irrelevance of the input variables in relation to the variable to be approximated is not taken into account. Regression problems share with the ordinal classification the existence of an explicit relationship of order between the values of the variable to be predicted. In a recent paper, the authors have proposed an ordinal classification algorithm that takes into account the detection of the irrelevance of input variables. This algorithm extracts a set of fuzzy rules from an example set, using as the basic model a sequential covering strategy along with a genetic algorithm. In this paper, a proposal for a regression algorithm based on this ordinal classification algorithm is presented. The proposed model can be interpreted as a multiclassifier and multilevel system that learns at each stage using the knowledge gained in previous stages. Due to similarities between regression and ordinal problems as well as the use of a set of ordinal algorithms, an error interval can be returned with the regression output value. Experimental results show the good behavior of the proposal as well as the results of the error interval.

[1]  J. Shaffer.  Modified Sequentially Rejective Multiple Test Procedures , 1986 .

[2]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[3]  D. Broomhead,et al.  Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks , 1988 .

[4]  David G. Kleinbaum,et al.  Logistic Regression. A Self- Learning Text , 1994 .

[5]  Jerry M. Mendel,et al.  Generating fuzzy rules by learning from examples , 1992, IEEE Trans. Syst. Man Cybern..

[6]  Francisco Herrera,et al.  A three-stage evolutionary process for learning descriptive and approximate fuzzy-logic-controller knowledge bases from examples , 1997, Int. J. Approx. Reason..

[7]  J. Rustagi.  Optimization Techniques in Statistics , 1994 .

[8]  Francisco Herrera,et al.  Hybridizing genetic algorithms with sharing scheme and evolution strategies for designing approximate fuzzy rule-based systems , 2001, Fuzzy Sets Syst..

[9]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[10]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[11]  Inés Couso,et al.  Fuzzy random variables-based modeling with GA-P algorithms , 2000 .

[12]  Jesús Alcalá-Fdez,et al.  Local identification of prototypes for genetic learning of accurate TSK fuzzy rule‐based systems , 2007, Int. J. Intell. Syst..

[13]  David García,et al.  Ordinal classification based on the sequential covering strategy , 2016, Int. J. Approx. Reason..

[14]  Francisco Herrera,et al.  A Two-stage Evolutionary Process for Designing Tsk Fuzzy Rule-based Systems a Two-stage Evolutionary Process for Designing Tsk Fuzzy Rule-based Systems , 1996 .

[15]  M. Sugeno,et al.  Structure identification of fuzzy model , 1988 .

[16]  Philip R. Thrift,et al.  Fuzzy Logic Synthesis with Genetic Algorithms , 1991, ICGA.

[17]  David García,et al.  Overview of the SLAVE learning algorithm: A review of its evolution and prospects , 2014, Int. J. Comput. Intell. Syst..

[18]  Michela Antonelli,et al.  An efficient multi-objective evolutionary fuzzy system for regression problems , 2013, Int. J. Approx. Reason..

[19]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[20]  Luciano Sánchez.  A random sets-based method for identifying fuzzy models , 1998, Fuzzy Sets Syst..

[21]  César Hervás-Martínez,et al.  Evolutionary product unit based neural networks for regression , 2006, Neural Networks.

[22]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[23]  Athanasios Tsakonas,et al.  A fuzzy evolutionary framework for combining ensembles , 2013, Appl. Soft Comput..

[24]  Inés Couso,et al.  Combining GP operators with SA search to evolve fuzzy rule based classifiers , 2001, Inf. Sci..

[25]  Pedro Antonio Gutiérrez,et al.  A preliminary study of ordinal metrics to guide a multi-objective evolutionary algorithm , 2011, 2011 11th International Conference on Intelligent Systems Design and Applications.

[26]  Raúl Rojas,et al.  Neural Networks - A Systematic Introduction , 1996 .

[27]  Rafael Alcalá,et al.  METSK-HDe: A multiobjective evolutionary algorithm to learn accurate TSK-fuzzy systems in high-dimensional and large-scale regression problems , 2014, Inf. Sci..

[28]  Raúl Pérez,et al.  Improving the genetic algorithm of SLAVE , 2009, SOCO 2009.

[29]  Ebrahim Mamdani,et al.  Applications of fuzzy algorithms for control of a simple dynamic plant , 1974 .

[30]  Nicolás García-Pedrajas,et al.  Nonlinear Boosting Projections for Ensemble Construction , 2007, J. Mach. Learn. Res..

[31]  A. Agresti,et al.  The analysis of ordered categorical data: An overview and a survey of recent developments , 2005 .

[32]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[33]  Abdollah Homaifar,et al.  Simultaneous design of membership functions and rule sets for fuzzy controllers using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[34]  P. McCullagh.  Regression Models for Ordinal Data , 1980 .

[35]  María José del Jesús,et al.  KEEL: a software tool to assess evolutionary algorithms for data mining problems , 2008, Soft Comput..

[36]  Geoff Holmes,et al.  Generating Rule Sets from Model Trees , 1999, Australian Joint Conference on Artificial Intelligence.

[37]  Christian Igel,et al.  Empirical evaluation of the improved Rprop learning algorithms , 2003, Neurocomputing.

[38]  Jung-Hua Wang,et al.  On the Internal Representations of Product Units , 2000, Neural Processing Letters.