Multivariate Versus Univariate Decision Trees

In this paper we present a new multivariate decision tree algorithm LMDT, which combines linear machines with decision trees. LMDT constructs each test in a linear machine and then eliminating irrelevant and noisy variables in a controlled manner. To examine LMDT''s ability to find good generalizations we present results for a variety of domains. We compare LMDT empirically to a univariate decision tree algorithm and observe that when multivariate tests are the appropriate bias for a given data set, LMDT finds small accurate trees.