Conic Relaxations for Semi-supervised Support Vector Machines

Semi-supervised support vector machines arise in machine learning as a model of mixed integer programming problem for classification. In this paper, we propose two convex conic relaxations for the original mixed integer programming problem. The first one is a new semi-definite relaxation, and its possibly maximal ratio of the optimal value is estimated approximately. The second one is a doubly nonnegative relaxation, which is relaxed from a well-known conic programming problem called completely positive programming problem that is equivalent to the original problem. Furthermore, we prove that the doubly nonnegative relaxation is tighter than the semi-definite relaxation. Finally, the numerical results show that two proposed relaxations not only generate proper classifiers but also outperform some existing methods in classification accuracy.

[1]  F. Tay,et al.  Application of support vector machines in financial time series forecasting , 2001 .

[2]  Dale Schuurmans,et al.  Maximum Margin Clustering , 2004, NIPS.

[3]  Charles R. Johnson,et al.  Topics in matrix analysis: The Hadamard product , 1991 .

[4]  F. Giannessi,et al.  Connections between Nonlinear Programming and Discrete Optimization , 1998 .

[5]  Ayhan Demiriz,et al.  Semi-Supervised Support Vector Machines , 1998, NIPS.

[6]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[7]  Samuel Burer,et al.  On the copositive representation of binary and continuous nonconvex quadratic programs , 2009, Math. Program..

[8]  Nai-Yang Deng,et al.  Support Vector Machines: Optimization Based Theory, Algorithms, and Extensions , 2012 .

[9]  Nello Cristianini,et al.  Convex Methods for Transduction , 2003, NIPS.

[10]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[11]  S. Sathiya Keerthi,et al.  Optimization Techniques for Semi-Supervised Support Vector Machines , 2008, J. Mach. Learn. Res..

[12]  Ivor W. Tsang,et al.  Tighter and Convex Maximum Margin Clustering , 2009, AISTATS.

[13]  Florian Jarre.  Burer’s key assumption for semidefinite and doubly nonnegative relaxations , 2012, Optim. Lett..

[14]  S. Sathiya Keerthi,et al.  Deterministic annealing for semi-supervised kernel machines , 2006, ICML.

[15]  Rong Jin,et al.  Generalized Maximum Margin Clustering and Unsupervised Kernel Learning , 2006, NIPS.

[16]  Alexander Zien,et al.  Semi-Supervised Classification by Low Density Separation , 2005, AISTATS.

[17]  Paul Tseng,et al.  Approximation Bounds for Quadratic Optimization with Homogeneous Quadratic Constraints , 2007, SIAM J. Optim..

[18]  Akiko Yoshise,et al.  On optimization over the doubly nonnegative cone , 2010, 2010 IEEE International Symposium on Computer-Aided Control System Design.

[19]  Yan-Qin Bai,et al.  Consensus Proximal Support Vector Machine for Classification Problems with Sparse Solutions , 2014 .

[20]  Zenglin Xu,et al.  Efficient Convex Relaxation for Transductive Support Vector Machine , 2007, NIPS.

[21]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[22]  Katta G. Murty,et al.  Some NP-complete problems in quadratic and nonlinear programming , 1987, Math. Program..

[23]  Samuel Burer,et al.  Optimizing a polyhedral-semidefinite relaxation of completely positive programs , 2010, Math. Program. Comput..

[24]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Wotao Yin,et al.  Alternating direction augmented Lagrangian methods for semidefinite programming , 2010, Math. Program. Comput..

[26]  Fei Wang,et al.  Cuts3vm: a fast semi-supervised svm algorithm , 2008, KDD.

[27]  Yanqin Bai,et al.  New SDP models for protein homology detection with semi-supervised SVM , 2013 .

[28]  Jason Weston,et al.  Large Scale Transductive SVMs , 2006, J. Mach. Learn. Res..

[29]  Oliver Kramer,et al.  Fast and simple gradient-based optimization for semi-supervised support vector machines , 2014, Neurocomputing.

[30]  Annabella Astorino,et al.  Support Vector Machine Polyhedral Separability in Semisupervised Learning , 2013, Journal of Optimization Theory and Applications.