Classification Trees With Unbiased Multiway Splits

Two univariate split methods and one linear combination split method are proposed for the construction of classification trees with multiway splits. Examples are given where the trees are more compact and hence easier to interpret than binary trees. A major strength of the univariate split methods is that they have negligible bias in variable selection, both when the variables differ in the number of splits they offer and when they differ in the number of missing values. This is an advantage because inferences from the tree structures can be adversely affected by selection bias. The new methods are shown to be highly competitive in terms of computational speed and classification accuracy of future observations.

[1]  H. Levene.  Robust tests for equality of variances , 1961 .

[2]  Peter Doyle,et al.  The Use of Automatic Interaction Detector and Similar Search Procedures , 1973 .

[3]  G. V. Kass.  An Exploratory Technique for Investigating Large Quantities of Categorical Data , 1980 .

[4]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[5]  Ivan Bratko,et al.  ASSISTANT 86: A Knowledge-Elicitation Tool for Sophisticated Users , 1987, EWSL.

[6]  W. Loh,et al.  Tree-Structured Classification Via Generalized Discriminant Analysis: Rejoinder , 1988 .

[7]  W. Loh,et al.  Tree-Structured Classification via Generalized Discriminant Analysis. , 1988 .

[8]  J. Ross Quinlan,et al.  Unknown Attribute Values in Induction , 1989, ML.

[9]  Douglas M. Hawkins,et al.  FIRM: Formal Inference-Based Recursive Modeling , 1991 .

[10]  Wray L. Buntine,et al.  Learning classification trees , 1992 .

[11]  Daryl Pregibon,et al.  Tree-based models , 1992 .

[12]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[13]  Wei-Yin Loh,et al.  Application of box-cox transformations to discrimination for the two-class problem , 1992 .

[14]  U. Fayyad.  On the induction of decision trees for multiple concept learning , 1991 .

[15]  Robin H. Lock.  1993 New Car Data , 1993 .

[16]  J. Ross Quinlan.  Unknown Attribute Values , 1993 .

[17]  R. Tibshirani,et al.  Flexible Discriminant Analysis by Optimal Scoring , 1994 .

[18]  Léopold Simar,et al.  Computer Intensive Methods in Statistics , 1994 .

[19]  Fritz Wysotzki,et al.  Automatic construction of decision trees for classification , 1994, Ann. Oper. Res..

[20]  Simon Kasif,et al.  A System for Induction of Oblique Decision Trees , 1994, J. Artif. Intell. Res..

[21]  Warren S. Sarle,et al.  Neural Networks and Statistical Models , 1994 .

[22]  Mary Rouncefield.  The Statistics of Poverty and Inequality , 1995 .

[23]  C. J. Stone,et al.  Polychotomous Regression , 1995 .

[24]  R. Tibshirani,et al.  Penalized Discriminant Analysis , 1995 .

[25]  R. Tibshirani,et al.  Discriminant Analysis by Gaussian Mixtures , 1996 .

[26]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[27]  W. Loh,et al.  SPLIT SELECTION METHODS FOR CLASSIFICATION TREES , 1997 .

[28]  H. Chipman,et al.  Bayesian CART Model Search , 1998 .

[29]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[30]  L. Breiman.  Arcing classifier (with discussion and a rejoinder by the author) , 1998 .

[31]  Heping Zhang.  Bayesian CART Model Search: Comment , 1998 .

[32]  W. Loh,et al.  REGRESSION TREES WITH UNBIASED VARIABLE SELECTION AND INTERACTION DETECTION , 2002 .