Self-advising support vector machine

The Support Vector Machine (SVM) is one of the most popular machine learning algorithms for classification and regression. SVM displays outstanding performance when utilized in many applications. However, different approaches have been proposed in order to improve its performance in general and in special cases. This paper proposes a new method of improving SVM performance in general. This method can be applied to all the types of SVMs that have differing kernel types. The key aim of the proposed approach is to transfer more information from the training phase to the testing phase. This information is obtained from the misclassified data of the training phase, which is ignored in the classic SVM. Experimental results from eleven datasets from the real online resources show that this approach can improve the classification performances of C-SVM and @n-SVM without adding any parameters to the learner algorithm. Statistical tests show significance in this improvement. The proposed method has been found to improve accuracies of C-SVM and @n-SVM in more than 67% of the experiments in which 11% of these improvements are more than 5%. It must be noted that the highest improvement found with this method was 25%.

[1]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[2]  Bernhard Schölkopf,et al.  Prior Knowledge in Support Vector Kernels , 1997, NIPS.

[3]  Wei Chu,et al.  An improved conjugate gradient scheme to the solution of least squares SVM , 2005, IEEE Transactions on Neural Networks.

[4]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[5]  Sheng-De Wang,et al.  Fuzzy support vector machines , 2002, IEEE Trans. Neural Networks.

[6]  Hyun-Chul Kim,et al.  Constructing support vector machine ensemble , 2003, Pattern Recognit..

[7]  Ruihu Wang,et al.  AdaBoost for Feature Selection, Classification and Its Relation with SVM, A Review , 2012 .

[8]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[9]  Ulrich H.-G. Kreßel,et al.  Pairwise classification and support vector machines , 1999 .

[10]  Fei-Yue Wang,et al.  Posterior probability support vector Machines for unbalanced data , 2005, IEEE Transactions on Neural Networks.

[11]  Gérard Bloch,et al.  Incorporating prior knowledge in support vector machines for classification: A review , 2008, Neurocomputing.

[12]  Dinggang Shen,et al.  An adaptive error penalization method for training an efficient and generalized SVM , 2006, Pattern Recognit..

[13]  Carlos García Garino,et al.  An autonomous labeling approach to support vector machines algorithms for network traffic anomaly detection , 2012, Expert Syst. Appl..

[14]  Gautam Kunapuli,et al.  Advice Refinement in Knowledge-Based Support Vector Machines , 2007 .

[15]  Bernhard Schölkopf,et al.  Training Invariant Support Vector Machines , 2002, Machine Learning.

[16]  K. Manimala,et al.  Optimization techniques for improving power quality data mining using wavelet packet based support vector machine , 2012, Neurocomputing.

[17]  Joarder Kamruzzaman,et al.  z-SVM: An SVM for Improved Classification of Imbalanced Data , 2006, Australian Conference on Artificial Intelligence.

[18]  Maya R. Gupta,et al.  Expected kernel for missing features in support vector machines , 2011, 2011 IEEE Statistical Signal Processing Workshop (SSP).

[19]  Shinto Eguchi,et al.  Boosting Method for Local Learning in Statistical Pattern Recognition , 2008, Neural Computation.

[20]  Jude W. Shavlik,et al.  Knowledge-Based Artificial Neural Networks , 1994, Artif. Intell..

[21]  Bo Peng,et al.  V-support Vector Classification with Uncertainty Based on Expert Advices , 2005, GrC.

[22]  Peter Williams,et al.  A Geometrical Method to Improve Performance of the Support Vector Machine , 2007, IEEE Transactions on Neural Networks.

[23]  José Luis Rojo-Álvarez,et al.  Feature selection using support vector machines and bootstrap methods for ventricular fibrillation detection , 2012, Expert Syst. Appl..

[24]  Vladimir Cherkassky,et al.  Vapnik-Chervonenkis (VC) learning theory and its applications , 1999 .

[25]  Jude W. Shavlik,et al.  Online Knowledge-Based Support Vector Machines , 2010, ECML/PKDD.

[26]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..