A new branch-and-bound approach to semi-supervised support vector machine

This paper develops a branch-and-bound algorithm to solve the 2-norm soft margin semi-supervised support vector machine. First, the original problem is reformulated as a non-convex quadratically constrained quadratic programming problem with a simple structure. Then, we propose a new lower bound estimator which is conceptually simple and easy to be implemented in the branch-and-bound scheme. Since this estimator preserves both a high efficiency and a relatively good quality in the convex relaxation, it leads to a high total efficiency in the whole computational process. The numerical tests on both artificial and real-world data sets demonstrate the better effectiveness and efficiency of this proposed approach, which is compared to other well-known methods on different semi-supervised support vector machine models.

[1]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[2]  S. Sathiya Keerthi,et al.  Deterministic annealing for semi-supervised kernel machines , 2006, ICML.

[3]  C. Ballantine.  On the Hadamard product , 1968 .

[4]  Congxin Wu,et al.  Membership evaluation and feature selection for fuzzy support vector machine based on fuzzy rough sets , 2011, Soft Comput..

[5]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[6]  Panos M. Pardalos,et al.  Quadratic programming with one negative eigenvalue is NP-hard , 1991, J. Glob. Optim..

[7]  Yanqin Bai,et al.  New SDP models for protein homology detection with semi-supervised SVM , 2013 .

[8]  Tijl De Bie,et al.  Semi-Supervised Learning Using Semi-Definite Programming , 2006, Semi-Supervised Learning.

[9]  Alain Biem,et al.  Semisupervised Least Squares Support Vector Machine , 2009, IEEE Transactions on Neural Networks.

[10]  F. Tay,et al.  Application of support vector machines in financial time series forecasting , 2001 .

[11]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[12]  M. Narasimha Murty,et al.  A fast quasi-Newton method for semi-supervised SVM , 2011, Pattern Recognit..

[13]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[14]  Ayhan Demiriz,et al.  Semi-Supervised Support Vector Machines , 1998, NIPS.

[15]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[17]  Xiaojin Zhu,et al.  Introduction to Semi-Supervised Learning , 2009, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[18]  Chao Wang,et al.  The support vector machine based on intuitionistic fuzzy number and kernel function , 2013, Soft Comput..

[19]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[20]  Fei Wang,et al.  Cuts3vm: a fast semi-supervised svm algorithm , 2008, KDD.

[21]  Charles R. Johnson,et al.  Topics in matrix analysis: The Hadamard product , 1991 .

[22]  Shu-Cherng Fang,et al.  FUZZY QUADRATIC SURFACE SUPPORT VECTOR MACHINE BASED ON FISHER DISCRIMINANT ANALYSIS , 2015 .

[23]  Stephen P. Boyd,et al.  Semidefinite Programming , 1996, SIAM Rev..

[24]  Cheng Lu,et al.  Convex reformulation for binary quadratic programming problems via average objective value maximization , 2014, Optimization Letters.

[25]  Alain Billionnet,et al.  Using a Mixed Integer Quadratic Programming Solver for the Unconstrained Quadratic 0-1 Problem , 2007, Math. Program..

[26]  Chao Wang,et al.  A new support vector machine based on type-2 fuzzy samples , 2013, Soft Comput..

[27]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[28]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[29]  Xiaowei Yang,et al.  A bilateral-truncated-loss based robust support vector machine for classification problems , 2015, Soft Comput..

[30]  Zenglin Xu,et al.  Efficient Convex Relaxation for Transductive Support Vector Machine , 2007, NIPS.

[31]  S. Sathiya Keerthi,et al.  Optimization Techniques for Semi-Supervised Support Vector Machines , 2008, J. Mach. Learn. Res..

[32]  Dale Schuurmans,et al.  Learning to Model Spatial Dependency: Semi-Supervised Discriminative Random Fields , 2006, NIPS.

[33]  Xin Yan,et al.  Conic Relaxations for Semi-supervised Support Vector Machines , 2016, J. Optim. Theory Appl..

[34]  Dale Schuurmans,et al.  Maximum Margin Clustering , 2004, NIPS.

[35]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[36]  Jason Weston,et al.  Large Scale Transductive SVMs , 2006, J. Mach. Learn. Res..

[37]  Oliver Kramer,et al.  Fast and simple gradient-based optimization for semi-supervised support vector machines , 2014, Neurocomputing.

[38]  Anton van den Hengel,et al.  Semidefinite Programming , 2014, Computer Vision, A Reference Guide.

[39]  Avrim Blum,et al.  Learning from Labeled and Unlabeled Data using Graph Mincuts , 2001, ICML.