Support vector machine with hypergraph-based pairwise constraints

Although support vector machine (SVM) has become a powerful tool for pattern classification and regression, a major disadvantage is it fails to exploit the underlying correlation between any pair of data points as much as possible. Inspired by the modified pairwise constraints trick, in this paper, we propose a novel classifier termed as support vector machine with hypergraph-based pairwise constraints to improve the performance of the classical SVM by introducing a new regularization term with hypergraph-based pairwise constraints (HPC). The new classifier is expected to not only learn the structural information of each point itself, but also acquire the prior distribution knowledge about each constrained pair by combining the discrimination metric and hypergraph learning together. Three major contributions of this paper can be summarized as follows: (1) acquiring the high-order relationships between different samples by hypergraph learning; (2) presenting a more reasonable discriminative regularization term by combining the discrimination metric and hypergraph learning; (3) improving the performance of the existing SVM classifier by introducing HPC regularization term. And the comprehensive experimental results on twenty-five datasets demonstrate the validity and advantage of our approach.

[1]  Aiguo Song,et al.  Improving clustering with pairwise constraints: a discriminative approach , 2012, Knowledge and Information Systems.

[2]  Charu C. Aggarwal,et al.  Towards graphical models for text processing , 2012, Knowledge and Information Systems.

[3]  Johan A. K. Suykens,et al.  Least squares support vector machine classifiers: a large scale algorithm , 1999 .

[4]  Ming Cheng,et al.  Combinative hypergraph learning for semi-supervised image classification , 2015, Neurocomputing.

[5]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[6]  Luiz Eduardo Soares de Oliveira,et al.  Pairwise fusion matrix for combining classifiers , 2007, Pattern Recognit..

[7]  Jue Wang,et al.  A general soft method for learning SVM classifiers with L1-norm penalty , 2008, Pattern Recognit..

[8]  Philippe Rigollet,et al.  Generalization Error Bounds in Semi-supervised Classification Under the Cluster Assumption , 2006, J. Mach. Learn. Res..

[9]  Changshui Zhang,et al.  Boosting with pairwise constraints , 2010, Neurocomputing.

[10]  Michael I. Jordan,et al.  A Robust Minimax Approach to Classification , 2003, J. Mach. Learn. Res..

[11]  Min Wu,et al.  Multi-label ensemble based on variable pairwise constraint projection , 2013, Inf. Sci..

[12]  Thomas G. Dietterich.  Adaptive computation and machine learning , 1998 .

[13]  Daniel S. Yeung,et al.  Structured large margin machines: sensitive to data distributions , 2007, Machine Learning.

[14]  Bernhard Schölkopf,et al.  Learning with Hypergraphs: Clustering, Classification, and Embedding , 2006, NIPS.

[15]  Václav Hlavác,et al.  An iterative algorithm learning the maximal margin classifier , 2003, Pattern Recognit..

[16]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[17]  S. Sathiya Keerthi,et al.  Improvements to Platt's SMO Algorithm for SVM Classifier Design , 2001, Neural Computation.

[18]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[19]  Daoqiang Zhang,et al.  Bagging Constraint Score for feature selection with pairwise constraints , 2010, Pattern Recognit..

[20]  Qiang Yang,et al.  Structural Regularized Support Vector Machine: A Framework for Structural Large Margin Classifier , 2011, IEEE Transactions on Neural Networks.

[21]  Xinbo Gao,et al.  Semi-supervised Gaussian process latent variable model with pairwise constraints , 2010, Neurocomputing.

[22]  Ming Yang,et al.  A novel hypothesis-margin based approach for feature selection with side pairwise constraints , 2010, Neurocomputing.

[23]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  David G. Stork,et al.  Pattern Classification , 1973 .

[25]  Jun Yu,et al.  Pairwise constraints based multiview features fusion for scene classification , 2013, Pattern Recognit..

[26]  Meng Wang,et al.  Adaptive Hypergraph Learning and its Application in Image Classification , 2012, IEEE Transactions on Image Processing.

[27]  TaoDacheng,et al.  Semi-supervised Gaussian process latent variable model with pairwise constraints , 2010 .

[28]  Marco Maggini,et al.  Learning from pairwise constraints by Similarity Neural Networks , 2012, Neural Networks.

[29]  Stephen J. Wright,et al.  Dissimilarity in Graph-Based Semi-Supervised Classification , 2007, AISTATS.

[30]  Qiang Qian,et al.  Multi-view classification with cross-view must-link and cannot-link side information , 2013, Knowl. Based Syst..

[31]  Zhe Wang,et al.  Matrixized learning machine with modified pairwise constraints , 2015, Pattern Recognit..

[32]  Joachim M. Buhmann,et al.  Learning with constrained and unlabelled data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[33]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[34]  Sergios Theodoridis,et al.  A geometric approach to Support Vector Machine (SVM) classification , 2006, IEEE Transactions on Neural Networks.

[35]  Mikhail Belkin,et al.  Manifold Regularization : A Geometric Framework for Learning from Examples , 2004 .

[36]  Federico Girosi,et al.  Support Vector Machines: Training and Applications , 1997 .

[37]  Nenghai Yu,et al.  Maximum Margin Clustering with Pairwise Constraints , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[38]  Michael R. Lyu,et al.  Learning large margin classifiers locally and globally , 2004, ICML.

[39]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[40]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.