A convex relaxation framework for a class of semi-supervised learning methods and its application in pattern recognition

Semi-supervised learning has been an attractive research tool for using unlabeled data in pattern recognition. Applying a novel semi-definite programming (SDP) relaxation strategy to a class of continuous semi-supervised support vector machines (S^3VMs), a new convex relaxation framework for the S^3VMs is proposed based on SDP. Compared with other SDP relaxations for S^3VMs, the proposed methods only require solving the primal problems and can implement L"1-norm regularization. Furthermore, the proposed technique is applied directly to recognize the purity of hybrid maize seeds using near-infrared spectral data, from which we find that the proposed method achieves equivalent performance to the exact solution algorithm for solving the S^3VM in different spectral regions. Experiments on several benchmark data sets demonstrate that the proposed convex technique is competitive with other SDP relaxation methods for solving semi-supervised SVMs in generalization.

[1]  Jason Weston,et al.  Large Scale Transductive SVMs , 2006, J. Mach. Learn. Res..

[2]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[3]  O. Mangasarian,et al.  Absolute value equations , 2006 .

[4]  Laisheng Wang,et al.  A class of smooth semi-supervised SVM by difference of convex functions programming and algorithm , 2013, Knowl. Based Syst..

[5]  Zenglin Xu,et al.  Efficient Convex Relaxation for Transductive Support Vector Machine , 2007, NIPS.

[6]  Xiaowei Yang,et al.  Local learning integrating global structure for large scale semi-supervised classification , 2013, Comput. Math. Appl..

[7]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[8]  Alexander Zien,et al.  Semi-Supervised Classification by Low Density Separation , 2005, AISTATS.

[9]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[10]  Junhui Wang,et al.  Efficient large margin semisupervised learning , 2007, AISTATS.

[11]  Nello Cristianini,et al.  Convex Methods for Transduction , 2003, NIPS.

[12]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[13]  S. Sathiya Keerthi,et al.  Optimization Techniques for Semi-Supervised Support Vector Machines , 2008, J. Mach. Learn. Res..

[14]  M. Narasimha Murty,et al.  A fast quasi-Newton method for semi-supervised SVM , 2011, Pattern Recognit..

[15]  Stephen P. Boyd,et al.  Semidefinite Programming , 1996, SIAM Rev..

[16]  S. Sathiya Keerthi,et al.  Branch and Bound for Semi-Supervised Support Vector Machines , 2006, NIPS.

[17]  Ayhan Demiriz,et al.  Semi-Supervised Support Vector Machines , 1998, NIPS.

[18]  Concha Bielza,et al.  Regularized logistic regression without a penalty term: An application to cancer classification with microarray data , 2011, Expert Syst. Appl..

[19]  Dale Schuurmans,et al.  Unsupervised and Semi-Supervised Multi-Class Support Vector Machines , 2005, AAAI.

[20]  Annabella Astorino,et al.  Nonsmooth Optimization Techniques for Semisupervised Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  M. Kojima,et al.  Second order cone programming relaxation of nonconvex quadratic optimization problems , 2001 .

[22]  O. Mangasarian,et al.  Semi-Supervised Support Vector Machines for Unlabeled Data Classification , 2001 .

[23]  Wei Pan,et al.  On Efficient Large Margin Semisupervised Learning: Method and Theory , 2009, J. Mach. Learn. Res..

[24]  Xueguang Shao,et al.  A wavelength selection method based on randomization test for near-infrared spectral analysis , 2009 .