论文信息 - Speeding Up Logistic Model Tree Induction

Speeding Up Logistic Model Tree Induction

Logistic Model Trees have been shown to be very accurate and compact classifiers [8]. Their greatest disadvantage is the computational complexity of inducing the logistic regression models in the tree. We address this issue by using the AIC criterion [1] instead of cross-validation to prevent overfitting these models. In addition, a weight trimming heuristic is used which produces a significant speedup. We compare the training time and accuracy of the new induction process with the original one on various datasets and show that the training time often decreases while the classification accuracy diminishes only slightly.

[1] J. Friedman. Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[2] Alberto Maria Segre,et al. Programs for Machine Learning , 1994 .

[3] Yoshua Bengio,et al. Inference for the Generalization Error , 1999, Machine Learning.

[4] Pat Langley,et al. Editorial: On Machine Learning , 1986, Machine Learning.

[5] H. Akaike,et al. Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[6] Ian H. Witten,et al. Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[7] J. Ross Quinlan,et al. C4.5: Programs for Machine Learning , 1992 .

[8] D. Ruppert. The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[9] Ian H. Witten,et al. Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[10] Yoav Freund,et al. Experiments with a New Boosting Algorithm , 1996, ICML.

[11] Eibe Frank,et al. Logistic Model Trees , 2003, Machine Learning.

[12] Ian Witten,et al. Data Mining , 2000 .

[13] Catherine Blake,et al. UCI Repository of machine learning databases , 1998 .

[14] Peter Bühlmann,et al. Boosting, model selection, lasso and nonnegative garrote , 2005 .