A GA-based model selection for smooth twin parametric-margin support vector machine

The recently proposed twin parametric-margin support vector machine, denoted by TPMSVM, gains good generalization and is suitable for many noise cases. However, in the TPMSVM, it solves two dual quadratic programming problems (QPPs). Moreover, compared with support vector machine (SVM), TPMSVM has at least four regularization parameters that need regulating, which affects its practical applications. In this paper, we increase the efficiency of TPMSVM from two aspects. First, by introducing a quadratic function, we directly optimize a pair of QPPs of TPMSVM in the primal space, called STPMSVM for short. Compared with solving two dual QPPs in the TPMSVM, STPMSVM can obviously improve the training speed without loss of generalization. Second, a genetic algorithm GA-based model selection for STPMSVM in the primal space is suggested. The GA-based STPMSVM can not only select the parameters efficiently, but also provide discriminative feature selection. Computational results on several synthetic as well as benchmark datasets confirm the great improvements on the training process of our GA-based STPMSVM.

[1]  Su-Yun Huang,et al.  Reduced Support Vector Machines: A Statistical Theory , 2007, IEEE Transactions on Neural Networks.

[2]  Hui Zhang,et al.  An integrated scheme for feature selection and parameter setting in the support vector machine modeling and its application to the prediction of pharmacokinetic properties of drugs , 2009, Artif. Intell. Medicine.

[3]  Yuh-Jye Lee,et al.  RSVM: Reduced Support Vector Machines , 2001, SDM.

[4]  Yuan-Hai Shao,et al.  An efficient support vector machine approach for identifying protein S-nitrosylation sites. , 2011, Protein and peptide letters.

[5]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[6]  Pei-Yi Hao,et al.  A new Support Vector classification algorithm with parametric-margin model , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[7]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[9]  David R. Musicant,et al.  Successive overrelaxation for support vector machines , 1999, IEEE Trans. Neural Networks.

[10]  Yuan-Hai Shao,et al.  Probabilistic outputs for twin support vector machines , 2012, Knowl. Based Syst..

[11]  Bernhard Schölkopf,et al.  Feature selection for support vector machines by means of genetic algorithm , 2003, Proceedings. 15th IEEE International Conference on Tools with Artificial Intelligence.

[12]  Olvi L. Mangasarian,et al.  Nonlinear Programming , 1969 .

[13]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[14]  Chih-Jen Lin,et al.  Training v-Support Vector Classifiers: Theory and Algorithms , 2001, Neural Computation.

[15]  Shigeo Abe,et al.  Support Vector Machines for Pattern Classification (Advances in Pattern Recognition) , 2005 .

[16]  Chih-Jen Lin,et al.  A Study on SMO-Type Decomposition Methods for Support Vector Machines , 2006, IEEE Transactions on Neural Networks.

[17]  Yuan-Hai Shao,et al.  Least squares recursive projection twin support vector machine for classification , 2012, Pattern Recognit..

[18]  Gilbert Syswerda,et al.  Uniform Crossover in Genetic Algorithms , 1989, ICGA.

[19]  Yuan-Hai Shao,et al.  Proximal parametric-margin support vector classifier and its applications , 2012, Neural Computing and Applications.

[20]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Bernhard Schölkopf,et al.  Support Vector Machine Applications in Computational Biology , 2004 .

[22]  Yuh-Jye Lee,et al.  SSVM: A Smooth Support Vector Machine for Classification , 2001, Comput. Optim. Appl..

[23]  Mustafa Ç. Pinar,et al.  On Smoothing Exact Penalty Functions for Convex Constrained Optimization , 1994, SIAM J. Optim..

[24]  Cheng-Lung Huang,et al.  A GA-based feature selection and parameters optimizationfor support vector machines , 2006, Expert Syst. Appl..

[25]  Glenn Fung,et al.  Multicategory Proximal Support Vector Machine Classifiers , 2005, Machine Learning.

[26]  Chao-Ton Su,et al.  Applying robust multi-response quality engineering for parameter selection using a novel neural-genetic algorithm , 2003, Comput. Ind..

[27]  Yuan-Hai Shao,et al.  A regularization for the projection twin support vector machine , 2013, Knowl. Based Syst..

[28]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[29]  Yuan-Hai Shao,et al.  Improvements on Twin Support Vector Machines , 2011, IEEE Transactions on Neural Networks.

[30]  David G. Stork,et al.  Pattern Classification , 1973 .

[31]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[32]  Xinjun Peng,et al.  TPMSVM: A novel twin parametric-margin support vector machine for pattern recognition , 2011, Pattern Recognit..

[33]  Yuan-Hai Shao,et al.  A coordinate descent margin based-twin support vector machine for classification , 2012, Neural Networks.

[34]  Shigeo Abe.  Support Vector Machines for Pattern Classification , 2010, Advances in Pattern Recognition.

[35]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[36]  Dino Isa,et al.  Text Document Preprocessing with the Bayes Formula for Classification Using the Support Vector Machine , 2008, IEEE Transactions on Knowledge and Data Engineering.

[37]  Madan Gopal,et al.  Application of smoothing technique on twin support vector machines , 2008, Pattern Recognit. Lett..

[38]  Theodore B. Trafalis,et al.  Support vector machine for regression and applications to financial forecasting , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[39]  Anirban Mukherjee,et al.  Cancer Classification from Gene Expression Data by NPPC Ensemble , 2011, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[40]  DebK.,et al.  A fast and elitist multiobjective genetic algorithm , 2002 .