Optimization Techniques for Semi-Supervised Support Vector Machines

Due to its wide applicability, the problem of semi-supervised classification is attracting increasing attention in machine learning. Semi-Supervised Support Vector Machines (S3VMs) are based on applying the margin maximization principle to both labeled and unlabeled examples. Unlike SVMs, their formulation leads to a non-convex optimization problem. A suite of algorithms have recently been proposed for solving S3VMs. This paper reviews key ideas in this literature. The performance and behavior of various S3VMs algorithms is studied together, under a common experimental setting.

[1]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[2]  Ron Cole,et al.  The ISOLET spoken letter database , 1990 .

[3]  Ayhan Demiriz,et al.  Semi-Supervised Support Vector Machines , 1998, NIPS.

[4]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[5]  Annabella Astorino,et al.  Nonsmooth Optimization Techniques for Semisupervised Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Jason Weston,et al.  Large Scale Transductive SVMs , 2006, J. Mach. Learn. Res..

[7]  Alexander Zien,et al.  A continuation method for semi-supervised SVMs , 2006, ICML.

[8]  Olivier Chapelle,et al.  Training a Support Vector Machine in the Primal , 2007, Neural Computation.

[9]  Zhijun Wu,et al.  The Eeective Energy Transformation Scheme as a General Continuation Approach to Global Optimization with Application to Molecular Conformation , 2022 .

[10]  Yoshua Bengio,et al.  Semi-supervised Learning by Entropy Minimization , 2004, CAP.

[11]  Thorsten Joachims,et al.  Transductive Support Vector Machines , 2006, Semi-Supervised Learning.

[12]  Tommi S. Jaakkola,et al.  Partially labeled classification with Markov random walks , 2001, NIPS.

[13]  Alan L. Yuille,et al.  The Concave-Convex Procedure , 2003, Neural Computation.

[14]  Alexander Zien,et al.  Semi-Supervised Classification by Low Density Separation , 2005, AISTATS.

[15]  Antônio de Pádua Braga,et al.  An evolutionary approach to transduction in support vector machines , 2005, Fifth International Conference on Hybrid Intelligent Systems (HIS'05).

[16]  S. Sathiya Keerthi,et al.  Deterministic annealing for semi-supervised kernel machines , 2006, ICML.

[17]  O. Mangasarian,et al.  Semi-Supervised Support Vector Machines for Unlabeled Data Classification , 2001 .

[18]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[19]  Guoping Wang,et al.  Learning with progressive transductive Support Vector Machine , 2002, 2002 IEEE International Conference on Data Mining, 2002. Proceedings..

[20]  Junhui Wang,et al.  On Transductive Support Vector Machines , 2006 .

[21]  Zoubin Ghahramani,et al.  Learning from labeled and unlabeled data with label propagation , 2002 .

[22]  Tijl De Bie,et al.  Semi-Supervised Learning Using Semi-Definite Programming , 2006, Semi-Supervised Learning.

[23]  Matthias Seeger,et al.  A Taxonomy for Semi-Supervised Learning Methods , 2006, Semi-Supervised Learning.

[24]  Dale Schuurmans,et al.  Maximum Margin Clustering , 2004, NIPS.

[25]  Mikhail Belkin,et al.  Beyond the point cloud: from transductive to semi-supervised learning , 2005, ICML.

[26]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[27]  S. Sathiya Keerthi,et al.  Large scale semi-supervised linear SVMs , 2006, SIGIR.

[28]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[29]  Sayan Mukherjee,et al.  Choosing Multiple Parameters for Support Vector Machines , 2002, Machine Learning.

[30]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[31]  Hong Liu,et al.  Fuzzy Transductive Support Vector Machines For Hypertext Classification , 2004, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[32]  Alexander Zien,et al.  Semi-Supervised Learning Using Semi-Definite Programming , 2006 .

[33]  S. Sathiya Keerthi,et al.  Branch and Bound for Semi-Supervised Support Vector Machines , 2006, NIPS.