An Effective SVM Ensemble Algorithm Based on Different Thresholds of PCA

This paper proposes an effective ensemble classifier, named PCAenSVM, which consists of ten weak Support Vector Machine classifiers based on different Principal Component Analysis thresholds. Those ten base Support Vector Machine classifiers are made up to fulfill classification tasks using Majority Voting strategy. Experiments are made on four UCI data sets and a data set from the Uppsala University to evaluate the performances of PCAenSVM. The results of PCAenSVM are compared with that of LibSVM and EnsembleSVM. Experimental results show that PCAenSVM has better classification accuracy than other two algorithms. Moreover, PCAenSVM has the same confidence level with the LibSVM, and its confidences of accuracy and sensitivity on those five data sets outperform that of the EnsembleSVM.

[1]  Daphne Koller,et al.  Support Vector Machine Active Learning with Applications to Text Classification , 2000, J. Mach. Learn. Res..

[2]  Ludmila I. Kuncheva,et al.  Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy , 2003, Machine Learning.

[3]  Lalit M. Patnaik,et al.  Classification of magnetic resonance brain images using wavelets as input to support vector machine and neural network , 2006, Biomed. Signal Process. Control..

[4]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[5]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.

[6]  Chung-Ho Hsieh,et al.  Novel solutions for an old disease: diagnosis of acute appendicitis with random forest, support vector machines, and artificial neural networks. , 2011, Surgery.

[7]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[8]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[10]  Lun-Ping Hung,et al.  A data driven ensemble classifier for credit scoring analysis , 2010, Expert Syst. Appl..

[11]  Chong Xu,et al.  GIS-based support vector machine modeling of earthquake-triggered landslide susceptibility in the Jianjiang River watershed, China , 2012 .

[12]  Ding-Xuan Zhou,et al.  SVM Soft Margin Classifiers: Linear Programming versus Quadratic Programming , 2005, Neural Computation.

[13]  Jessica J. Fridrich,et al.  Ensemble Classifiers for Steganalysis of Digital Media , 2012, IEEE Transactions on Information Forensics and Security.

[14]  Hyun-Chul Kim,et al.  Pattern classification using support vector machine ensemble , 2002, Object recognition supported by user interaction for service robots.

[15]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[16]  Johan A. K. Suykens,et al.  EnsembleSVM: a library for ensemble learning using support vector machines , 2014, J. Mach. Learn. Res..

[17]  Sheng-De Wang,et al.  Fuzzy support vector machines , 2002, IEEE Trans. Neural Networks.

[18]  Adrian D. C. Chan,et al.  Myoelectric Control Development Toolbox , 2007 .

[19]  Zhiwen Yu,et al.  Identifying Protein-Kinase-Specific Phosphorylation Sites Based on the Bagging–AdaBoost Ensemble Approach , 2010, IEEE Transactions on NanoBioscience.

[20]  Ning Wang,et al.  AdaBoost classifiers for pecan defect classification , 2011 .

[21]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[22]  Thomas G. Dietterich.  Machine-Learning Research , 1997, AI Mag..