Handling continuous attributes in an evolutionary inductive learner

This work analyzes experimentally discretization algorithms for handling continuous attributes in evolutionary learning. We consider a learning system that induces a set of rules in a fragment of first-order logic (evolutionary inductive logic programming), and introduce a method where a given discretization algorithm is used to generate initial inequalities, which describe subranges of attributes' values. Mutation operators exploiting information on the class label of the examples (supervised discretization) are used during the learning process for refining inequalities. The evolutionary learning system is used as a platform for testing experimentally four algorithms: two variants of the proposed method, a popular supervised discretization algorithm applied prior to induction, and a discretization method which does not use information on the class labels of the examples (unsupervised discretization). Results of experiments conducted on artificial and real life datasets suggest that the proposed method provides an effective and robust technique for handling continuous attributes by means of inequalities.

[1]  A. Debnath,et al.  Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. Correlation with molecular orbital energies and hydrophobicity. , 1991, Journal of medicinal chemistry.

[2]  Pier Luca Lanzi,et al.  An Analysis of Generalization in the XCS Classifier System , 1999, Evolutionary Computation.

[3]  Federico Divina,et al.  Non-universal Suffrage Selection Operators Favor Population Diversity in Genetic Algorithms , 2003, GECCO.

[4]  Ramón López de Mántaras,et al.  A distance-based attribute selection measure for decision tree induction , 1991, Machine Learning.

[5]  Saso Dzeroski,et al.  Detecting Traffic Problems with ILP , 1998, ILP.

[6]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[7]  Jorma Rissanen,et al.  Stochastic Complexity in Statistical Inquiry , 1989, World Scientific Series in Computer Science.

[8]  Saso Dzeroski,et al.  ILP Experiments in Detecting Traffic Problems , 1998, ECML.

[9]  Randy Kerber,et al.  ChiMerge: Discretization of Numeric Attributes , 1992, AAAI.

[10]  Ron Kohavi,et al.  Supervised and Unsupervised Discretization of Continuous Features , 1995, ICML.

[11]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[12]  Jesús S. Aguilar-Ruiz,et al.  Natural Coding: A More Efficient Representation for Evolutionary Learning , 2003, GECCO.

[13]  Keki B. Irani,et al.  Multi-interval discretization of continuos attributes as pre-processing for classi cation learning , 1993, IJCAI 1993.

[14]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[15]  Stewart W. Wilson.  Generalization in the XCS Classifier System , 1998 .

[16]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[17]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[18]  Hendrik Blockeel,et al.  Top-Down Induction of First Order Logical Decision Trees , 1998, AI Commun..

[19]  Ian H. Witten,et al.  Weka machine learning algorithms in java , 2000 .

[20]  Cezary Z. Janikow,et al.  A knowledge-intensive genetic algorithm for supervised learning , 1993, Machine Learning.

[21]  Jesús S. Aguilar-Ruiz,et al.  Supervised learning by means of accuracy-aware evolutionary algorithms , 2003, Inf. Sci..

[22]  Jesús S. Aguilar-Ruiz.  Removing examples and discovering Hierarchical Decision Rules: With Evolutionary Algorithms , 2001, AI Commun..

[23]  Stephen Muggleton,et al.  Learning from Positive Data , 1996, Inductive Logic Programming Workshop.

[24]  Filippo Neri,et al.  Search-Intensive Concept Induction , 1995, Evolutionary Computation.

[25]  Federico Divina,et al.  Evolutionary Concept Learning , 2002, GECCO.

[26]  Luc De Raedt,et al.  Top-down induction of logical decision trees , 1997 .

[27]  Wim Van Laer.  From Propositional to First Order Logic in Machine Learning and Data Mining - Induction of first ord , 2002 .

[28]  D. Kibler,et al.  Instance-based learning algorithms , 2004, Machine Learning.

[29]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[30]  Dr. Alex A. Freitas.  Data Mining and Knowledge Discovery with Evolutionary Algorithms , 2002, Natural Computing Series.

[32]  Usama M. Fayyad,et al.  On the Handling of Continuous-Valued Attributes in Decision Tree Generation , 1992, Machine Learning.

[33]  Jesús S. Aguilar-Ruiz,et al.  Discretization oriented to Decision Rules Generation , 2001 .

[34]  K. De Jong,et al.  Using Genetic Algorithms for Concept Learning , 2004, Machine Learning.

[35]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[36]  Hussein A. Abbass,et al.  Be real! XCS with continuous-valued inputs , 2005, GECCO '05.

[37]  Bart Demoen,et al.  Improving the Efficiency of Inductive Logic Programming Through the Use of Query Packs , 2011, J. Artif. Intell. Res..

[38]  Ron Kohavi,et al.  Error-Based and Entropy-Based Discretization of Continuous Features , 1996, KDD.

[39]  Miguel Toro,et al.  Evolutionary learning of hierarchical decision rules , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[40]  Xavier Llorà,et al.  Inducing Partially-Defined Instances with Evolutionary Algorithms , 2001, ICML.

[41]  Chandrika Kamath,et al.  Inducing oblique decision trees with evolutionary algorithms , 2003, IEEE Trans. Evol. Comput..

[42]  Jaume Bacardit,et al.  Evolving Multiple Discretizations with Adaptive Intervals for a Pittsburgh Rule-Based Learning Classifier System , 2003, GECCO.

[43]  Peter Clark,et al.  Rule Induction with CN2: Some Recent Improvements , 1991, EWSL.

[44]  Marek Kretowski,et al.  An Evolutionary Algorithm Using Multivariate Discretization for Decision Rule Induction , 1999, PKDD.

[45]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[46]  Jaume Bacardit,et al.  Evolution of Multi-adaptive Discretization Intervals for a Rule-Based Genetic Learning System , 2002, IBERAMIA.

[47]  Gilles Venturini,et al.  Learning First Order Logic Rules with a Genetic Algorithm , 1995, KDD.

[48]  Luc De Raedt,et al.  Inductive Constraint Logic , 1995, ALT.

[49]  Robert C. Holte,et al.  Very Simple Classification Rules Perform Well on Most Commonly Used Datasets , 1993, Machine Learning.

[50]  Federico Divina,et al.  A Method for Handling Numerical Attributes in GA-Based Inductive Concept Learners , 2003, GECCO.

[51]  Huan Liu,et al.  Discretization: An Enabling Technique , 2002, Data Mining and Knowledge Discovery.

[52]  Xavier Llorà,et al.  Knowledge-independent data mining with fine-grained parallel evolutionary algorithms , 2001 .

[53]  Alex Alves Freitas,et al.  Mining Very Large Databases with Parallel Processing , 1997, The Kluwer International Series on Advances in Database Systems.

[54]  Saso Dzeroski,et al.  EXPERIMENTS IN PREDICTING BIODEGRADABILITY , 1999, ILP.

[55]  Jason Catlett,et al.  On Changing Continuous Attributes into Ordered Discrete Attributes , 1991, EWSL.

[56]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.