Texture Based Look-Ahead for Decision-Tree Induction

Decision tree induction corresponds to partitioning the input space through the use of internal nodes. A node attempts to classify the instances misclassified by upper level nodes using a greedy search strategy, i.e. to maximize the number of instances correctly classified at the node. However, in so doing it is possible that the distribution of the remaining (misclassified) instances is complex thereby requiring a large number of additional nodes for classification. The so-called look-ahead method was proposed to examine the classifiability of the remaining instances. Typically, this is done using a classifier parameterized identically (e.g. hyperplane) to the one used for partitioning. The number of correct classification achieved in the look-ahead stage is factored in choosing the partition established at a node. This has led to mixed results. In this paper we argue that 1-step (or a few step) look-ahead amounts to an approximate estimate of the classifiability of the remaining instances. In many cases, it is gross approximation and hence the results are mixed. We present an alternative approach to look-ahead which uses the joint probability of occurrence of the classes as a way of characterizing the classifiability of the remaining instances. We present the algorithm and some experimental results.