Bi-density twin support vector machines for pattern recognition

In this paper we present a classifier called bi-density twin support vector machines (BDTWSVMs) for data classification. In the training stage, BDTWSVMs first compute the relative density degrees for all training points using the intra-class graph whose weights are determined by a local scaling heuristic strategy, then optimize a pair of nonparallel hyperplanes through two smaller sized support vector machine (SVM)-typed problems. In the prediction stage, BDTWSVMs assign to the class label depending on the kernel density degree-based distances from each test point to the two hyperplanes. BDTWSVMs not only inherit good properties from twin support vector machines (TWSVMs) but also give good description for data points. The experimental results on toy as well as publicly available datasets indicate that BDTWSVMs compare favorably with classical SVMs and TWSVMs in terms of generalization.

[1]  Dirk P. Kroese,et al.  Kernel density estimation via diffusion , 2010, 1011.2602.

[2]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[3]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[4]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Xinjun Peng,et al.  Primal twin support vector regression and its sparse approximation , 2010, Neurocomputing.

[6]  Songcan Chen,et al.  MultiK-MHKS: A Novel Multiple Kernel Learning Algorithm , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[8]  Xinjun Peng,et al.  Efficient twin parametric insensitive support vector regression model , 2012, Neurocomputing.

[9]  Yuan-Hai Shao,et al.  Improvements on Twin Support Vector Machines , 2011, IEEE Transactions on Neural Networks.

[10]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  M. Omair Ahmad,et al.  Optimizing the kernel in the empirical feature space , 2005, IEEE Transactions on Neural Networks.

[12]  Chunxia Zhao,et al.  Localized twin SVM via convex minimization , 2011, Neurocomputing.

[13]  Suresh Chandra,et al.  Reduced twin support vector regression , 2011, Neurocomputing.

[14]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[15]  Xinjun Peng,et al.  A nu-twin support vector machine (nu-TSVM) classifier and its geometric algorithms , 2010, Inf. Sci..

[16]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[17]  Bin Chen,et al.  Proximal support vector machine using local information , 2009, Neurocomputing.

[18]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[19]  Anirban Mukherjee,et al.  Nonparallel plane proximal classifier , 2009, Signal Process..

[20]  Anirban Mukherjee,et al.  Newton's method for nonparallel plane proximal classifier with unity norm hyperplanes , 2010, Signal Process..

[21]  J. Simonoff.  Smoothing Methods in Statistics , 1998 .

[22]  Masashi Sugiyama,et al.  Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis , 2007, J. Mach. Learn. Res..

[23]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[24]  Xinjun Peng,et al.  TPMSVM: A novel twin parametric-margin support vector machine for pattern recognition , 2011, Pattern Recognit..

[25]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[27]  Xinjun Peng,et al.  Building sparse twin support vector machine classifiers in primal space , 2011, Inf. Sci..

[28]  Madan Gopal,et al.  Application of smoothing technique on twin support vector machines , 2008, Pattern Recognit. Lett..

[29]  Qiang Yang,et al.  Discriminatively regularized least-squares classification , 2009, Pattern Recognit..

[30]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[31]  Nando de Freitas,et al.  Sequential Monte Carlo Methods in Practice , 2001, Statistics for Engineering and Information Science.

[32]  E. Lehmann.  Model Specification: The Views of Fisher and Neyman, and Later Developments , 1990 .

[33]  Xinjun Peng,et al.  TSVR: An efficient Twin Support Vector Machine for regression , 2010, Neural Networks.