Fisher's decision tree

Univariate decision trees are classifiers currently used in many data mining applications. This classifier discovers partitions in the input space via hyperplanes that are orthogonal to the axes of attributes, producing a model that can be understood by human experts. One disadvantage of univariate decision trees is that they produce complex and inaccurate models when decision boundaries are not orthogonal to axes. In this paper we introduce the Fisher's Tree, it is a classifier that takes advantage of dimensionality reduction of Fisher's linear discriminant and uses the decomposition strategy of decision trees, to come up with an oblique decision tree. Our proposal generates an artificial attribute that is used to split the data in a recursive way. The Fisher's decision tree induces oblique trees whose accuracy, size, number of leaves and training time are competitive with respect to other decision trees reported in the literature. We use more than ten public available data sets to demonstrate the effectiveness of our method.

[1]  David G. Stork,et al.  Pattern Classification , 1973 .

[2]  Sunil Vadera,et al.  Inducing safer oblique trees without costs , 2005, Expert Syst. J. Knowl. Eng..

[3]  Mohammad Reza Kangavari,et al.  Using genetic programming for the induction of oblique decision trees , 2007, ICMLA 2007.

[4]  Denis Pomorski,et al.  Inductive learning of decision trees: application to fault isolation of an induction motor , 2001 .

[5]  King-Sun Fu,et al.  A Nonparametric Partitioning Procedure for Pattern Classification , 1969, IEEE Transactions on Computers.

[6]  F. Mayer-Lindenberg,et al.  LEONARDO - The computational intelligence (CI) model selection wizard , 2007, ICMLA 2007.

[7]  Huan Liu,et al.  A connectionist approach to generating oblique decision trees , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[8]  Wang Xiaohu,et al.  An Application of Decision Tree Based on ID3 , 2012 .

[9]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[10]  Chandrika Kamath,et al.  Inducing oblique decision trees with evolutionary algorithms , 2003, IEEE Trans. Evol. Comput..

[11]  Witold Pedrycz,et al.  Data Mining: A Knowledge Discovery Approach , 2007 .

[12]  João Gama,et al.  Oblique Linear Tree , 1997, IDA.

[13]  R. Schapire.  The Strength of Weak Learnability , 1990, Machine Learning.

[14]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[15]  Carla E. Brodley,et al.  Linear Machine Decision Trees , 1991 .

[16]  Naresh Manwani,et al.  A Geometric Algorithm for Learning Oblique Decision Trees , 2009, PReMI.

[17]  João Gama,et al.  Discriminant Trees , 1999, ICML.

[18]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[19]  Ethem Alpaydin,et al.  Linear Discriminant Trees , 2000, ICML.

[20]  Pei-Chann Chang,et al.  A hybrid model combining case-based reasoning and fuzzy decision tree for medical data classification , 2011, Appl. Soft Comput..

[21]  George H. John,et al.  Robust Linear Discriminant Trees , 1995, AISTATS.

[22]  Ping He,et al.  Tree classifier in singular vertor space , 2008, 2008 International Conference on Machine Learning and Cybernetics.

[23]  Nicandro Cruz-Ramírez,et al.  Discovering interobserver variability in the cytodiagnosis of breast cancer using decision trees and Bayesian networks , 2009, Appl. Soft Comput..

[24]  Alex Alves Freitas,et al.  A Survey of Evolutionary Algorithms for Decision-Tree Induction , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[25]  P. Shanti Sastry,et al.  New algorithms for learning and pruning oblique decision trees , 1999, IEEE Trans. Syst. Man Cybern. Part C.

[26]  Sunil Vadera,et al.  CSNL: A cost-sensitive non-linear decision tree algorithm , 2010, TKDD.

[27]  V. Menkovski,et al.  Oblique Decision Trees using embedded Support Vector Machines in classifier ensembles , 2008, 2008 7th IEEE International Conference on Cybernetic Intelligent Systems.

[28]  Xiao-Bai Li.  A scalable decision tree system and its application in pattern recognition and intrusion detection , 2005 .

[29]  Vijay S. Iyengar.  HOT: heuristics for oblique trees , 1999, Proceedings 11th International Conference on Tools with Artificial Intelligence.

[30]  Muh-Cherng Wu,et al.  An effective application of decision tree to stock trading , 2006, Expert Syst. Appl..

[31]  Simon Kasif,et al.  A System for Induction of Oblique Decision Trees , 1994, J. Artif. Intell. Res..