Structural twin parametric-margin support vector machine for binary classification

Twin parametric-margin support vector machine (TPMSVM) determines the more flexible parametric-margin hyperplanes through a pair of quadratic programming problems (QPPs) compared with classical support vector machine (SVM). However, it ignores the prior structural information in data. In this paper, we present a structural twin parametric-margin support vector machine (STPMSVM) for classification. The two optimization problems of STPMSVM focus on the structural information of the corresponding classes based on the cluster granularity, which is vital for designing a good classifier in different real-world problems. Furthermore, two Mahalanobis distances are respectively introduced into its corresponding QPPs based on the structural information. STPMSVM has a special case of TPMSVM when each ellipsoid cluster is a unit ball in a reproducing kernel Hilbert space. Experimental results demonstrate that STPMSVM is often superior in generalization performance to other learning algorithms.

[1]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Daniel S. Yeung,et al.  Structured large margin machines: sensitive to data distributions , 2007, Machine Learning.

[3]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[4]  Silvio Borer,et al.  Normalization in Support Vector Machines , 2001, DAGM-Symposium.

[5]  Philippe Rigollet,et al.  Generalization Error Bounds in Semi-supervised Classification Under the Cluster Assumption , 2006, J. Mach. Learn. Res..

[6]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[7]  J. Mercer.  Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations , 1909 .

[8]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[9]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  J. H. Ward.  Hierarchical Grouping to Optimize an Objective Function , 1963 .

[11]  Anirban Mukherjee,et al.  Nonparallel plane proximal classifier , 2009, Signal Process..

[12]  Dong Xu,et al.  Robust minimum class variance twin support vector machine classifier , 2011, Neural Computing and Applications.

[13]  Xinjun Peng,et al.  TSVR: An efficient Twin Support Vector Machine for regression , 2010, Neural Networks.

[14]  Xinjun Peng,et al.  Building sparse twin support vector machine classifiers in primal space , 2011, Inf. Sci..

[15]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[16]  Xinjun Peng,et al.  A nu-twin support vector machine (nu-TSVM) classifier and its geometric algorithms , 2010, Inf. Sci..

[17]  Jian Yang,et al.  Recursive projection twin support vector machine via within-class variance minimization , 2011, Pattern Recognit..

[18]  Xinjun Peng,et al.  TPMSVM: A novel twin parametric-margin support vector machine for pattern recognition , 2011, Pattern Recognit..

[19]  Dong Xu,et al.  Twin Mahalanobis distance-based support vector machines for pattern recognition , 2012, Inf. Sci..

[20]  Maurice K. Wong,et al.  Algorithm AS136: A k-means clustering algorithm. , 1979 .

[21]  Michael R. Lyu,et al.  Maxi–Min Margin Machine: Learning Large Margin Classifiers Locally and Globally , 2008, IEEE Transactions on Neural Networks.

[22]  Xinjun Peng,et al.  Primal twin support vector regression and its sparse approximation , 2010, Neurocomputing.

[23]  Nikolas P. Galatsanos,et al.  A support vector machine approach for detection of microcalcifications , 2002, IEEE Transactions on Medical Imaging.

[24]  Xinjun Peng,et al.  Efficient twin parametric insensitive support vector regression model , 2012, Neurocomputing.

[25]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[26]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[27]  Michael I. Jordan,et al.  A Robust Minimax Approach to Classification , 2003, J. Mach. Learn. Res..

[28]  Yuan-Hai Shao,et al.  Probabilistic outputs for twin support vector machines , 2012, Knowl. Based Syst..

[29]  Qiang Yang,et al.  Structural Regularized Support Vector Machine: A Framework for Structural Large Margin Classifier , 2011, IEEE Transactions on Neural Networks.

[30]  Yuan-Hai Shao,et al.  A regularization for the projection twin support vector machine , 2013, Knowl. Based Syst..

[31]  Bernhard Schölkopf,et al.  Kernel Methods in Computational Biology , 2005 .

[32]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[33]  King-Sun Fu,et al.  A Sentence-to-Sentence Clustering Procedure for Pattern Analysis , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[34]  Madan Gopal,et al.  Application of smoothing technique on twin support vector machines , 2008, Pattern Recognit. Lett..

[35]  Dong Xu,et al.  A twin-hypersphere support vector machine classifier and the fast learning algorithm , 2013, Inf. Sci..