SPLIT SELECTION METHODS FOR CLASSIFICATION TREES

Classification trees based on exhaustive search algorithms tend to be bi- ased towards selecting variables that afford more splits. As a result, such trees should be interpreted with caution. This article presents an algorithm called QUEST that has negligible bias. Its split selection strategy shares similarities with the FACT method, but it yields binary splits and the final tree can be selected by a direct stopping rule or by pruning. Real and simulated data are used to compare QUEST with the exhaustive search approach. QUEST is shown to be substantially faster and the size and classification accuracy of its trees are typically comparable to those of exhaustive search. A classification tree is a rule for predicting the class of an object from the values of its predictor variables. The tree is constructed by recursively parti- tioning a learning sample of data in which the class label and the values of the predictor variables for each case are known. Each partition is represented by a node in the tree. Two approaches to split selection have been proposed in the statistical liter- ature. The first and more popular approach examines all possible binary splits of the data along each predictor variable to select the split that most reduces some measure of node impurity. It is used, for example, by the THAID (Morgan and Sonquist (1963), Morgan and Messenger (1973)) and CART (Breiman, Friedman, Olshen and Stone (1984)) algorithms. If X is an ordered variable, this approach searches over all possible values c for splits of the form X ≤ c. (1)