Laplacian smooth twin support vector machine for semi-supervised classification

Laplacian twin support vector machine (Lap-TSVM) is a state-of-the-art nonparallel-planes semi-supervised classifier. It tries to exploit the geometrical information embedded in unlabeled data to boost its generalization ability. However, Lap-TSVM may endure heavy burden in training procedure since it needs to solve two quadratic programming problems (QPPs) with the matrix “inversion” operation. In order to enhance the performance of Lap-TSVM, this paper presents a new formulation of Lap-TSVM, termed as Lap-STSVM. Rather than solving two QPPs in dual space, firstly, we convert the primal constrained QPPs of Lap-TSVM into unconstrained minimization problems (UMPs). Afterwards, a smooth technique is introduced to make these UMPs twice differentiable. At last, a fast Newton–Armijo algorithm is designed to solve the UMPs in Lap-STSVM. Experimental evaluation on both artificial and real-world datasets demonstrate the benefits of the proposed approach.

[1]  Yong Shi,et al.  Laplacian twin support vector machine for semi-supervised classification , 2012, Neural Networks.

[2]  David R. Musicant,et al.  Lagrangian Support Vector Machines , 2001, J. Mach. Learn. Res..

[3]  Anuj Karpatne,et al.  Twin support vector regression for the simultaneous learning of a function and its derivatives , 2012, International Journal of Machine Learning and Cybernetics.

[4]  Yuh-Jye Lee,et al.  SSVM: A Smooth Support Vector Machine for Classification , 2001, Comput. Optim. Appl..

[5]  Yuan-Hai Shao,et al.  Least squares recursive projection twin support vector machine for classification , 2012, Pattern Recognit..

[6]  Thorsten Joachims,et al.  Learning to classify text using support vector machines - methods, theory and algorithms , 2002, The Kluwer international series in engineering and computer science.

[7]  Zhi-Hua Zhou,et al.  New Semi-Supervised Classification Method Based on Modified Cluster Assumption , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Madan Gopal,et al.  Application of smoothing technique on twin support vector machines , 2008, Pattern Recognit. Lett..

[9]  Qiang Yang,et al.  Discriminatively regularized least-squares classification , 2009, Pattern Recognit..

[10]  Tu-Bao Ho,et al.  Detecting disease genes based on semi-supervised learning and protein-protein interaction networks , 2012, Artif. Intell. Medicine.

[11]  Ujjwal Maulik,et al.  A novel semisupervised SVM for pixel classification of remote sensing imagery , 2012, Int. J. Mach. Learn. Cybern..

[12]  Jian Yang,et al.  Smooth twin support vector regression , 2010, Neural Computing and Applications.

[13]  Xinjun Peng,et al.  Building sparse twin support vector machine classifiers in primal space , 2011, Inf. Sci..

[14]  Yuan-Hai Shao,et al.  Multiple birth support vector machine for multi-class classification , 2012, Neural Computing and Applications.

[15]  Nong Sang,et al.  Using clustering analysis to improve semi-supervised classification , 2013, Neurocomputing.

[16]  Changsheng Xu,et al.  Boosted multi-class semi-supervised learning for human action recognition , 2011, Pattern Recognit..

[17]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[18]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[19]  Yuan-Hai Shao,et al.  Improving Lap-TSVM with Successive Overrelaxation and Differential Evolution , 2013, ITQM.

[20]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[21]  Xinjun Peng,et al.  TPMSVM: A novel twin parametric-margin support vector machine for pattern recognition , 2011, Pattern Recognit..

[22]  Yuan-Hai Shao,et al.  Improvements on Twin Support Vector Machines , 2011, IEEE Transactions on Neural Networks.

[23]  Mikhail Belkin,et al.  Laplacian Support Vector Machines Trained in the Primal , 2009, J. Mach. Learn. Res..

[24]  Samuel Kotz,et al.  On the Student's t-distribution and the t-statistic , 2007 .

[25]  Yuan-Hai Shao,et al.  A GA-based model selection for smooth twin parametric-margin support vector machine , 2013, Pattern Recognit..

[26]  Walmir M. Caminhas,et al.  A review of machine learning approaches to Spam filtering , 2009, Expert Syst. Appl..

[27]  Huanhuan Chen,et al.  Semisupervised Classification With Cluster Regularization , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[28]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Ying-Ke Lei,et al.  Semi-supervised locally discriminant projection for classification and recognition , 2011, Knowl. Based Syst..