论文信息 - Comments on A Parallel Mixture of SVMs for Very Large Scale Problems

Comments on A Parallel Mixture of SVMs for Very Large Scale Problems

Collobert, Bengio, and Bengio (2002) recently introduced a novel approach to using a neural network to provide a class prediction from an ensemble of support vector machines (SVMs). This approach has the advantage that the required computation scales well to very large data sets. Experiments on the Forest Cover data set show that this parallel mixture is more accurate than a single SVM, with 90.72 accuracy reported on an independent test set. Although this accuracy is impressive, their article does not consider alternative types of classifiers. We show that a simple ensemble of decision trees results in a higher accuracy, 94.75, and is computationally efficient. This result is somewhat surprising and illustrates the general value of experimental comparisons using different types of classifiers.

Lawrence O. Hall | Kevin W. Bowyer | Xiaomei Liu

[1] Bernhard Schölkopf,et al. Training Invariant Support Vector Machines , 2002, Machine Learning.

[2] Nitesh V. Chawla,et al. Distributed learning with bagging-like performance , 2003, Pattern Recognit. Lett..

[3] J. Ross Quinlan,et al. C4.5: Programs for Machine Learning , 1992 .

[4] Catherine Blake,et al. UCI Repository of machine learning databases , 1998 .

[5] Lawrence O. Hall,et al. A New Ensemble Diversity Measure Applied to Thinning Ensembles , 2003, Multiple Classifier Systems.

[6] Denis J. Dean,et al. Comparative accuracies of artificial neural networks and discriminant analysis in predicting forest cover types from cartographic variables , 1999 .

[7] Samy Bengio,et al. A Parallel Mixture of SVMs for Very Large Scale Problems , 2001, Neural Computation.

[8] G DietterichThomas. An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees , 2000 .

[9] Vladimir N. Vapnik,et al. The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[10] Thomas G. Dietterich. An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization , 2000, Machine Learning.