The Alternating Decision Tree Learning Algorithm

The application of boosting procedures to decision tree algorithms has been shown to produce very accurate classi ers. These classiers are in the form of a majority vote over a number of decision trees. Unfortunately, these classi ers are often large, complex and diÆcult to interpret. This paper describes a new type of classi cation rule, the alternating decision tree, which is a generalization of decision trees, voted decision trees and voted decision stumps. At the same time classi ers of this type are relatively easy to interpret. We present a learning algorithm for alternating decision trees that is based on boosting. Experimental results show it is competitive with boosted decision tree algorithms such as C5.0, and generates rules that are usually smaller in size and thus easier to interpret. In addition these rules yield a natural measure of classi cation con dence which can be used to improve the accuracy at the cost of abstaining from predicting examples that are hard to classify.

[1]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[2]  Thomas G. Dietterich,et al.  Pruning Adaptive Boosting , 1997, ICML.

[3]  Yishay Mansour,et al.  On the Boosting Ability of Top-Down Decision Tree Learning Algorithms , 1999, J. Comput. Syst. Sci..

[4]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[5]  Pedro M. Domingos.  Knowledge Acquisition from Examples Via Multiple Models , 1997 .

[6]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[7]  J. Ross Quinlan,et al.  Bagging, Boosting, and C4.5 , 1996, AAAI/IAAI, Vol. 1.

[8]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[9]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[10]  Ron Kohavi,et al.  Option Decision Trees with Majority Votes , 1997, ICML.

[11]  Wray L. Buntine,et al.  Learning classification trees , 1992 .

[12]  Mark Craven,et al.  Extracting comprehensible models from trained neural networks , 1996 .

[13]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[14]  Leo Breiman,et al.  Bias, Variance , And Arcing Classifiers , 1996 .

[15]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[16]  Y. Freund,et al.  Discussion of the Paper \additive Logistic Regression: a Statistical View of Boosting" By , 2000 .