论文信息 - Shrinking classification trees for bootstrap aggregation

Shrinking classification trees for bootstrap aggregation

Abstract Bootstrap aggregating (bagging) classification trees has been shown to improve the accuracy over using a single classification tree. However, it has been noticed that bagging pruned trees does not necessarily result in better performance than bagging unpruned trees does. It is our goal here to discuss this issue in the context of shrinking, instead of pruning, trees. Due to many duplicated observations in a bootstrap sample, the usual shrinking determined by cross-validation (CV) in bagging is so conservative that the resulting shrunken tree is not much different from the unshrunken tree, leading to their close performance. We propose to choose the shrinkage parameter for each base tree in bagging by using only extra-bootstrap observations as test cases. For the digit data taken from Breiman et al. (1984), we find that our proposal leads to improved accuracy over that from bagging unshrunken trees.

Wei Pan | W. Pan

[1] Wray L. Buntine,et al. Learning classification trees , 1992 .

[2] Philip A. Chou,et al. Optimal Partitioning for Classification and Regression Trees , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[3] Daryl Pregibon,et al. Tree-based models , 1992 .

[4] B. Efron. Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation , 1983 .

[5] R. Tibshirani,et al. Improvements on Cross-Validation: The 632+ Bootstrap Method , 1997 .

[6] L. Breiman. Pasting Bites Together For Prediction In Large Data Sets And On-Line , 1996 .

[7] Robert Tibshirani,et al. An Introduction to the Bootstrap , 1994 .

[8] Allan R. Wilks,et al. The new S language: a programming environment for data analysis and graphics , 1988 .

[9] Yoav Freund,et al. A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[10] Leo Breiman,et al. Bagging Predictors , 1996, Machine Learning.

[11] Lalit R. Bahl,et al. A tree-based statistical language model for natural language speech recognition , 1989, IEEE Trans. Acoust. Speech Signal Process..