An overview on semi-supervised support vector machine

Support vector machine (SVM) is a machine learning method based on statistical learning theory. It has a lot of advantages, such as solid theoretical foundation, global optimization, the sparsity of the solution, nonlinear and generalization. The standard form of SVM only applies to supervised learning. Large amount of data generated in real life is unlabeled, and the standard form of SVM cannot make good use of these data to improve its learning ability. However, semi-supervised support vector machine (S3VM) is a good solution to this problem. This paper reviews the recent progress in semi-supervised support vector machine. First, the basic theory of S3VM is expounded and discussed in detail; then, the mainstream model of S3VM is presented, including transductive support vector machine, Laplacian support vector machine, S3VM training via the label mean, S3VM based on cluster kernel; finally, we give the conclusions and look ahead to the research on S3VM.

[1]  Le Thi Hoai An,et al.  Sparse semi-supervised support vector machines by DC programming and DCA , 2015, Neurocomputing.

[2]  Ye Wang,et al.  Training TSVM with the proper number of positive samples , 2005, Pattern Recognit. Lett..

[3]  Shifei Ding,et al.  Incremental Learning Algorithm for Support Vector Data Description , 2011, J. Softw..

[4]  Gongping Yang,et al.  On Co-Training Style Algorithms , 2008, 2008 Fourth International Conference on Natural Computation.

[5]  Mikhail Belkin,et al.  Maximum Margin Semi-Supervised Learning for Structured Variables , 2005, NIPS 2005.

[6]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[7]  Friedhelm Schwenker,et al.  Semi-supervised Learning , 2013, Handbook on Neural Information Processing.

[8]  Xiaojin Zhu,et al.  Semi-Supervised Learning , 2010, Encyclopedia of Machine Learning.

[9]  Zhi-Hua Zhou,et al.  Semi-supervised learning using label mean , 2009, ICML '09.

[10]  Jing Yang,et al.  A Transductive Support Vector Machine Algorithm Based on Spectral Clustering , 2012 .

[11]  Mohamed Cheriet,et al.  Genetic algorithm–based training for semi-supervised SVM , 2010, Neural Computing and Applications.

[12]  Rui Zhang,et al.  Semi-Supervised Hyperspectral Image Classification Using Spatio-Spectral Laplacian Support Vector Machine , 2014, IEEE Geoscience and Remote Sensing Letters.

[13]  Yong Luo,et al.  Multiview Vector-Valued Manifold Regularization for Multilabel Image Classification , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Luis Gómez-Chova,et al.  Semisupervised Image Classification With Laplacian Support Vector Machines , 2008, IEEE Geoscience and Remote Sensing Letters.

[15]  Jinglu Hu,et al.  A Transductive Support Vector Machine with adjustable quasi-linear kernel for semi-supervised data classification , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[16]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[17]  Xiaojin Zhu,et al.  Introduction to Semi-Supervised Learning , 2009, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[18]  Rui Zhang,et al.  Least Square Transduction Support Vector Machine , 2009, Neural Processing Letters.

[19]  Peng Liang,et al.  Explore Semi-supervised Support Vector Machine Algorithm for the Application of Physical Education Effect Evaluation , 2012 .

[20]  Ke Lu,et al.  Semi-supervised Support Vector Learning for Face Recognition , 2006, ISNN.

[21]  David J. Miller,et al.  A Mixture of Experts Classifier with Learning Based on Both Labelled and Unlabelled Data , 1996, NIPS.

[22]  Sheng Ding,et al.  Spectral and Wavelet-based Feature Selection with Particle Swarm Optimization for Hyperspectral Classification , 2011, J. Softw..

[23]  J. Benediktsson,et al.  Semi-Supervised Self Learning for Hyperspectral Image Classification , 2012 .

[24]  Bernhard Schölkopf,et al.  Cluster Kernels for Semi-Supervised Learning , 2002, NIPS.

[25]  R. Bharat Rao,et al.  Bayesian Co-Training , 2007, J. Mach. Learn. Res..

[26]  Zhang Li-wen,et al.  Appropriateness in Applying SVMs to Text Classification , 2010 .

[27]  Michael I. Jordan,et al.  Supervised Learning and Divide-and-Conquer: A Statistical Approach , 1993, ICML.

[28]  Mikhail Belkin,et al.  Laplacian Support Vector Machines Trained in the Primal , 2009, J. Mach. Learn. Res..

[29]  M. Narasimha Murty,et al.  A fast quasi-Newton method for semi-supervised SVM , 2011, Pattern Recognit..

[30]  S. V. N. Vishwanathan,et al.  A Quasi-Newton Approach to Nonsmooth Convex Optimization Problems in Machine Learning , 2008, J. Mach. Learn. Res..

[31]  Chen Yi.  A Progressive Transductive Inference Algorithm Based on Support Vector Machine , 2003 .

[32]  Ayhan Demiriz,et al.  Semi-Supervised Support Vector Machines , 1998, NIPS.

[33]  Wang Xi-li.  Semi-supervised SVM classification method based on cluster kernel , 2013 .

[34]  Irena Koprinska,et al.  Co-training using RBF Nets and Different Feature Splits , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[35]  Jason Weston,et al.  Large Scale Transductive SVMs , 2006, J. Mach. Learn. Res..

[36]  Li-Xin Ding,et al.  Lp Norm Constraint Multi-Kernel Learning Method for Semi-Supervised Support Vector: Lp Norm Constraint Multi-Kernel Learning Method for Semi-Supervised Support Vector , 2014 .

[37]  Oliver Kramer,et al.  Fast and simple gradient-based optimization for semi-supervised support vector machines , 2014, Neurocomputing.

[38]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[39]  Todd,et al.  Diffuse large B-cell lymphoma outcome prediction by gene-expression profiling and supervised machine learning , 2002, Nature Medicine.

[40]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[41]  Xuelong Li,et al.  Multitraining Support Vector Machine for Image Retrieval , 2006, IEEE Transactions on Image Processing.

[42]  Hu Qing.  L_p Norm Constraint Multi-Kernel Learning Method for Semi-Supervised Support Vector Machine , 2013 .

[43]  Tong Zhang,et al.  The Value of Unlabeled Data for Classification Problems , 2000, ICML 2000.

[44]  Cordelia Schmid,et al.  Multimodal semi-supervised learning for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[45]  Xuelong Li,et al.  Supervised Tensor Learning , 2005, ICDM.

[46]  Zhao Hong-hai.  Semi-Supervised Support Vector Machines for Data Classification , 2004 .

[47]  Qian Lin-jie.  Semisupervised Classification of Hyperspectral Image Based on Clustering Kernel and LS-SVM , 2011 .

[48]  Alexander Zien,et al.  A continuation method for semi-supervised SVMs , 2006, ICML.

[49]  Stéphane Canu,et al.  A multiple kernel framework for inductive semi-supervised SVM learning , 2012, Neurocomputing.

[50]  Qiya Su,et al.  A new semi-supervised support vector machine classifier based on wavelet transform and its application in the iris image recognition , 2014 .

[51]  Alexander Zien,et al.  Semi-Supervised Classification by Low Density Separation , 2005, AISTATS.

[52]  Lina Yao,et al.  A New Classification Method Based on Semi-supervised Support Vector Machine , 2014, HCC.

[53]  Oliver Kramer,et al.  Sparse Quasi-Newton Optimization for Semi-supervised Support Vector Machines , 2012, ICPRAM.

[54]  Gustavo Camps-Valls,et al.  Semisupervised Remote Sensing Image Classification With Cluster Kernels , 2009, IEEE Geoscience and Remote Sensing Letters.

[55]  Tijl De Bie,et al.  Semi-Supervised Learning Using Semi-Definite Programming , 2006, Semi-Supervised Learning.

[56]  Tao Xin-min,et al.  The SVM Classification Algorithm Based on Semi-Supervised Gauss Mixture Model Kernel , 2013 .

[57]  Ke Tang,et al.  Combining Semi-Supervised and active learning for hyperspectral image classification , 2013, 2013 IEEE Symposium on Computational Intelligence and Data Mining (CIDM).

[58]  Yang Zhou,et al.  Rapid identification between edible oil and swill-cooked dirty oil by using a semi-supervised support vector machine based on graph and near-infrared spectroscopy , 2015 .

[59]  Tommi S. Jaakkola,et al.  Partially labeled classification with Markov random walks , 2001, NIPS.

[60]  Habibollah Haron,et al.  Semi-supervised SVM-based Feature Selection for Cancer Classification using Microarray Gene Expression Data , 2015, IEA/AIE.

[61]  Xian-Sheng Hua,et al.  Ensemble Manifold Regularization , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Yuanqing Li,et al.  A Self-Training Semi-Supervised Support Vector Machine Algorithm and its Applications in Brain Computer Interface , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[63]  Zhiqiang Zhang,et al.  Laplacian p-norm proximal support vector machine for semi-supervised classification , 2014, Neurocomputing.

[64]  Francisco Herrera,et al.  On the characterization of noise filters for self-training semi-supervised in nearest neighbor classification , 2014, Neurocomputing.

[65]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[66]  Jon Atli Benediktsson,et al.  Semisupervised Self-Learning for Hyperspectral Image Classification , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[67]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[68]  S. Sathiya Keerthi,et al.  Optimization Techniques for Semi-Supervised Support Vector Machines , 2008, J. Mach. Learn. Res..

[69]  Rama Chellappa,et al.  Structure From Planar Motion , 2006, IEEE Transactions on Image Processing.

[70]  S. Sathiya Keerthi,et al.  Branch and Bound for Semi-Supervised Support Vector Machines , 2006, NIPS.

[71]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[72]  Yong Shi,et al.  Cost-Sensitive Support Vector Machine for Semi-Supervised Learning , 2013, ICCS.

[73]  Zhi-Hua Zhou,et al.  Towards Making Unlabeled Data Never Hurt , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[74]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[75]  Qi Bing-juan,et al.  An Overview on Theory and Algorithm of Support Vector Machines , 2011 .

[76]  Yong Luo,et al.  Manifold Regularized Multitask Learning for Semi-Supervised Multilabel Image Classification , 2013, IEEE Transactions on Image Processing.

[77]  Yong Shi,et al.  Successive Overrelaxation for Laplacian Support Vector Machine , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[78]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[79]  Zhi-Hua Zhou,et al.  Cost-Sensitive Semi-Supervised Support Vector Machine , 2010, AAAI.

[80]  S. Sathiya Keerthi,et al.  Deterministic annealing for semi-supervised kernel machines , 2006, ICML.

[81]  Ding Shi-fei.  Advances of Support Vector Machines(SVM) , 2011 .

[82]  Yansheng Lu,et al.  Markov random field based fusion for supervised and semi-supervised multi-modal image classification , 2014, Multimedia Tools and Applications.