A kernel-free quadratic surface support vector machine for semi-supervised learning

In this paper, we propose a kernel-free semi-supervised quadratic surface support vector machine model for binary classification. The model is formulated as a mixed-integer programming problem, which is equivalent to a non-convex optimization problem with absolute-value constraints. Using the relaxation techniques, we derive a semi-definite programming problem for semi-supervised learning. By solving this problem, the proposed model is tested on some artificial and public benchmark data sets. Preliminary computational results indicate that the proposed method outperforms some existing well-known methods for solving semi-supervised support vector machine with a Gaussian kernel in terms of classification accuracy.

[1]  Issam Dagher,et al.  Quadratic kernel-free non-linear support vector machine , 2008, J. Glob. Optim..

[2]  Ralf Stecking,et al.  Support vector machines for classifying and describing credit applicants: detecting typical and critical regions , 2005, J. Oper. Res. Soc..

[3]  Le Thi Hoai An,et al.  DCA Based Algorithms for Feature Selection in Semi-supervised Support Vector Machines , 2013, MLDM.

[4]  David J. Hand,et al.  Direct versus indirect credit scoring classifications , 2002, J. Oper. Res. Soc..

[5]  S. Sathiya Keerthi,et al.  Deterministic annealing for semi-supervised kernel machines , 2006, ICML.

[6]  Xin Yan,et al.  Conic Relaxations for Semi-supervised Support Vector Machines , 2016, J. Optim. Theory Appl..

[7]  Rong Jin,et al.  Generalized Maximum Margin Clustering and Unsupervised Kernel Learning , 2006, NIPS.

[8]  J. Suykens,et al.  Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research , 2015, Eur. J. Oper. Res..

[9]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Wotao Yin,et al.  Alternating direction augmented Lagrangian methods for semidefinite programming , 2010, Math. Program. Comput..

[11]  James V. Hansen,et al.  Some evidence on forecasting time-series with support vector machines , 2006, J. Oper. Res. Soc..

[12]  Nello Cristianini,et al.  Convex Methods for Transduction , 2003, NIPS.

[13]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[14]  Jian Luo,et al.  Quadratic Surface Support Vector Machines with Applications. , 2014 .

[15]  Alexander Zien,et al.  Semi-Supervised Classification by Low Density Separation , 2005, AISTATS.

[16]  S. Sathiya Keerthi,et al.  Optimization Techniques for Semi-Supervised Support Vector Machines , 2008, J. Mach. Learn. Res..

[17]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[18]  Yanqin Bai,et al.  New SDP models for protein homology detection with semi-supervised SVM , 2013 .

[19]  Hui Li,et al.  Imbalance-oriented SVM methods for financial distress prediction: a comparative study among the new SB-SVM-ensemble method and traditional methods , 2014, J. Oper. Res. Soc..

[20]  Jason Weston,et al.  Large Scale Transductive SVMs , 2006, J. Mach. Learn. Res..

[21]  Ayhan Demiriz,et al.  Semi-Supervised Support Vector Machines , 1998, NIPS.

[22]  Sebastián Maldonado,et al.  A Semi-supervised Approach for Reject Inference in Credit Scoring Using SVMs , 2010, ICDM.

[23]  Shu-Cherng Fang,et al.  FUZZY QUADRATIC SURFACE SUPPORT VECTOR MACHINE BASED ON FISHER DISCRIMINANT ANALYSIS , 2015 .

[24]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[25]  Zenglin Xu,et al.  Efficient Convex Relaxation for Transductive Support Vector Machine , 2007, NIPS.

[26]  Dale Schuurmans,et al.  Unsupervised and Semi-Supervised Multi-Class Support Vector Machines , 2005, AAAI.

[27]  Nai-Yang Deng,et al.  Support Vector Machines: Optimization Based Theory, Algorithms, and Extensions , 2012 .

[28]  Aris A. Syntetos,et al.  Classification for forecasting and stock control: a case study , 2008, J. Oper. Res. Soc..

[29]  Fei Wang,et al.  Cuts3vm: a fast semi-supervised svm algorithm , 2008, KDD.

[30]  Johan A. K. Suykens,et al.  Benchmarking state-of-the-art classification algorithms for credit scoring , 2003, J. Oper. Res. Soc..

[31]  Kenneth Kennedy,et al.  Using semi-supervised classifiers for credit scoring , 2013, J. Oper. Res. Soc..