论文信息 - Improving Random Forests

Improving Random Forests

Random forests are one of the most successful ensemble methods which exhibits performance on the level of boosting and support vector machines. The method is fast, robust to noise, does not overfit and offers possibilities for explanation and visualization of its output. We investigate some possibilities to increase strength or decrease correlation of individual trees in the forest. Using several attribute evaluation measures instead of just one gives promising results. On the other hand replacement of ordinary voting with voting weighted with margin achieved on most similar instances gives improvements which are statistically highly significant over several data sets.

Marko Robnik-Sikonja | M. Robnik-Sikonja

[1] Leo Breiman,et al. Bagging Predictors , 1996, Machine Learning.

[2] J. Ross Quinlan,et al. C4.5: Programs for Machine Learning , 1992 .

[3] Yoav Freund,et al. Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[4] Igor Kononenko,et al. On Biases in Estimating Multi-Valued Attributes , 1995, IJCAI.

[5] Catherine Blake,et al. UCI Repository of machine learning databases , 1998 .

[6] Igor Kononenko,et al. Estimating Attributes: Analysis and Extensions of RELIEF , 1994, ECML.

[7] Yoav Freund,et al. Experiments with a New Boosting Algorithm , 1996, ICML.

[8] Aiko M. Hormann,et al. Programs for Machine Learning. Part I , 1962, Inf. Control..

[9] Leo Breiman,et al. Random Forests , 2001, Machine Learning.

[10] Marko Robnik-Sikonja,et al. Theoretical and Empirical Analysis of ReliefF and RReliefF , 2003, Machine Learning.

[11] Thomas G. Dietterich,et al. Applying the Waek Learning Framework to Understand and Improve C4.5 , 1996, ICML.

[12] David J. Hand,et al. A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems , 2001, Machine Learning.

[13] Leo Breiman,et al. Classification and Regression Trees , 1984 .

[14] Kurt Hornik,et al. The support vector machine under test , 2003, Neurocomputing.