Structural regularized projection twin support vector machine for data classification

Abstract Projection twin support vector machine (PTSVM) seeks two projection directions for two classes by solving two smaller-sized quadratic programming problems (QPPs), such that the projected samples of one class are well separated from those of the other one in its respective subspace. However, it only simply considers the prior class-based structural information in the optimization problems. In this paper, a structural regularized PTSVM (SRPTSVM) classifier for binary classification is presented. This proposed SRPTSVM focuses on the cluster-based structural information of the corresponding class in each optimization problem, which is vital for designing a good classifier in different real-world problems. This SRPTSVM is extended to a nonlinear version by the kernel trick. Experimental results demonstrate that SRPTSVM is superior in generalization performance to other classifiers.

[1]  Daniel S. Yeung,et al.  Structured large margin machines: sensitive to data distributions , 2007, Machine Learning.

[2]  Michael I. Jordan,et al.  A Robust Minimax Approach to Classification , 2003, J. Mach. Learn. Res..

[3]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  King-Sun Fu,et al.  A Sentence-to-Sentence Clustering Procedure for Pattern Analysis , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[5]  Qiang Yang,et al.  Structural Regularized Support Vector Machine: A Framework for Structural Large Margin Classifier , 2011, IEEE Transactions on Neural Networks.

[6]  Ning Ye,et al.  Multi-weight vector projection support vector machines , 2010, Pattern Recognit. Lett..

[7]  Michael R. Lyu,et al.  Maxi–Min Margin Machine: Learning Large Margin Classifiers Locally and Globally , 2008, IEEE Transactions on Neural Networks.

[8]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Pannagadatta K. Shivaswamy.  Ellipsoidal Kernel Machines , 2007 .

[10]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[11]  Yuan-Hai Shao,et al.  A regularization for the projection twin support vector machine , 2013, Knowl. Based Syst..

[12]  Bernhard Schölkopf,et al.  Kernel Methods in Computational Biology , 2005 .

[13]  Mikhail Belkin,et al.  Manifold Regularization : A Geometric Framework for Learning from Examples , 2004 .

[14]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Xinjun Peng,et al.  Efficient twin parametric insensitive support vector regression model , 2012, Neurocomputing.

[16]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[17]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[18]  Nikolas P. Galatsanos,et al.  A support vector machine approach for detection of microcalcifications , 2002, IEEE Transactions on Medical Imaging.

[19]  J. H. Ward.  Hierarchical Grouping to Optimize an Objective Function , 1963 .

[20]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[21]  Philippe Rigollet,et al.  Generalization Error Bounds in Semi-supervised Classification Under the Cluster Assumption , 2006, J. Mach. Learn. Res..

[22]  Xinjun Peng,et al.  TPMSVM: A novel twin parametric-margin support vector machine for pattern recognition , 2011, Pattern Recognit..

[23]  Anirban Mukherjee,et al.  Nonparallel plane proximal classifier , 2009, Signal Process..

[24]  Xinjun Peng,et al.  TSVR: An efficient Twin Support Vector Machine for regression , 2010, Neural Networks.

[25]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[26]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[27]  Xinjun Peng,et al.  A nu-twin support vector machine (nu-TSVM) classifier and its geometric algorithms , 2010, Inf. Sci..

[28]  Jian Yang,et al.  Recursive projection twin support vector machine via within-class variance minimization , 2011, Pattern Recognit..

[29]  Madan Gopal,et al.  Application of smoothing technique on twin support vector machines , 2008, Pattern Recognit. Lett..

[30]  Philip Chan,et al.  Determining the number of clusters/segments in hierarchical clustering/segmentation algorithms , 2004, 16th IEEE International Conference on Tools with Artificial Intelligence.

[31]  Dong Xu,et al.  A twin-hypersphere support vector machine classifier and the fast learning algorithm , 2013, Inf. Sci..