A Global Search Approach for Inducing Oblique Decision Trees Using Differential Evolution

This paper describes the application of a Differential Evolution based approach for inducing oblique decision trees in a global search strategy. By using both the number of attributes and the number of class labels in a dataset, this approach determines the size of the real-valued vector utilized for encoding the set of hyperplanes used as test conditions in the internal nodes of an oblique decision tree. Also a scheme of three steps to map the linear representation of candidate solutions into feasible oblique decision trees is described. Experimental results obtained show that this approach induces more accurate classifiers than those produced by other proposed induction methods.

[1]  Jun Yu Li,et al.  Differential evolution-based parameters optimisation and feature selection for support vector machine , 2016, IEEE CSE 2016.

[2]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[3]  Simon Kasif,et al.  OC1: A Randomized Induction of Oblique Decision Trees , 1993, AAAI.

[4]  Alex Alves Freitas,et al.  Evolving decision trees with beam search-based initialization and lexicographic multi-objective evaluation , 2014, Inf. Sci..

[5]  R. Struharik,et al.  Evolving full oblique decision trees , 2015, 2015 16th IEEE International Symposium on Computational Intelligence and Informatics (CINTI).

[6]  J. Ross Quinlan,et al.  Simplifying Decision Trees , 1987, Int. J. Man Mach. Stud..

[7]  Antonio J. Nebro,et al.  jMetal: A Java framework for multi-objective optimization , 2011, Adv. Eng. Softw..

[8]  Francisco Herrera,et al.  Evolutionary-based selection of generalized instances for imbalanced classification , 2012, Knowl. Based Syst..

[9]  Chun-Gui Xu,et al.  A genetic programming-based approach to the classification of multiclass microarray datasets , 2009, Bioinform..

[10]  D. Hawkins.  The Problem of Overfitting , 2004 .

[11]  P. V. G. Bradbeer,et al.  The Construction and Evaluation of Decision Trees: a Comparison of Evolutionary and Concept Learning Methods , 1997, Evolutionary Computing, AISB Workshop.

[12]  Tea Tuÿsar.  Optimizing Accuracy and Size of Decision Trees , 2007 .

[13]  H. Khanna Nehemiah,et al.  Neural network classifier optimization using Differential Evolution with Global Information and Back Propagation algorithm for clinical datasets , 2016, Appl. Soft Comput..

[14]  S. Santhosh Baboo,et al.  An Empirical Model for Thyroid Disease Classification using Evolutionary Multivariate Bayseian Prediction Method , 2016 .

[15]  Ivanoe De Falco,et al.  Differential Evolution for automatic rule extraction from medical databases , 2013, Appl. Soft Comput..

[16]  G. Tutz,et al.  An introduction to recursive partitioning: rationale, application, and characteristics of classification and regression trees, bagging, and random forests. , 2009, Psychological methods.

[17]  Francisco Herrera,et al.  A Survey on the Application of Genetic Programming to Classification , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[18]  Xin Yao,et al.  Multiobjective genetic programming for maximizing ROC performance , 2014, Neurocomputing.

[19]  Frederico G. Guimarães,et al.  Differential Evolution and Perceptron Decision Trees for Classification Tasks , 2012, IDEAL.

[20]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[21]  Marek Kretowski,et al.  Evolutionary Learning of Linear Trees with Embedded Feature Selection , 2006, ICAISC.

[22]  Ronald L. Rivest,et al.  Constructing Optimal Binary Decision Trees is NP-Complete , 1976, Inf. Process. Lett..

[23]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[24]  Martijn C. J. Bot.  Improving Induction of Linear Classification Trees with Genetic Programming , 2000, GECCO.

[25]  Alexandros Agapitos,et al.  Maximum Margin Decision Surfaces for Increased Generalisation in Evolutionary Decision Tree Learning , 2011, EuroGP.

[26]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[27]  Swagatam Das,et al.  Automatic Clustering Using an Improved Differential Evolution Algorithm , 2007 .

[28]  Dimitris K. Tasoulis,et al.  A Review of Major Application Areas of Differential Evolution , 2008 .

[29]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[30]  Alex Alves Freitas,et al.  A Survey of Evolutionary Algorithms for Decision-Tree Induction , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[31]  K. Hornik,et al.  Unbiased Recursive Partitioning: A Conditional Inference Framework , 2006 .

[32]  Chandrika Kamath,et al.  Inducing oblique decision trees with evolutionary algorithms , 2003, IEEE Trans. Evol. Comput..

[33]  Christian Veenhuis,et al.  Tree Based Differential Evolution , 2009, EuroGP.