Structural nonparallel support vector machine for pattern recognition

It has been widely accepted that the underlying structural information in the training data within classes is significant for a good classifier in real-world problems. However, existing structural classifiers do not balance structural information's relationships both intra-class and inter-class. Combining the structural information with nonparallel support vector machine (NPSVM), we design a new structural nonparallel support vector machine (called SNPSVM). Each model of SNPSVM considers not only the compactness in both classes by the structural information but also the separability between classes, thus it can fully exploit prior knowledge to directly improve the algorithm's generalization capacity. Furthermore, we apply the improved alternating direction method of multipliers (ADMM) to SNPSVM. Both our model itself and the solving algorithm can guarantee that it can deal with large-scale classification problems with a huge number of instances as well as features. Experimental results show that SNPSVM is superior to the other current algorithms based on structural information of data in both computation time and classification accuracy. HighlightsWe design a new structural nonparallel support vector machine (SNPSVM).SNPSVM can fully exploit prior knowledge in the datasets.We apply the alternating direction method of multipliers (ADMM) for SNPSVM.We apply the block and parallel techniques in our algorithms.

[1]  Yingjie Tian,et al.  Large-scale linear nonparallel support vector machine solver , 2014, Neurocomputing.

[2]  Yuan-Hai Shao,et al.  Least squares recursive projection twin support vector machine for classification , 2012, Pattern Recognit..

[3]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[4]  Xinjun Peng,et al.  TPMSVM: A novel twin parametric-margin support vector machine for pattern recognition , 2011, Pattern Recognit..

[5]  Yong Shi,et al.  Structural twin support vector machine for classification , 2013, Knowl. Based Syst..

[6]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  C C TsangEric,et al.  Structured large margin machines , 2007 .

[8]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[9]  Dong Xu,et al.  Structural twin parametric-margin support vector machine for binary classification , 2013, Knowl. Based Syst..

[10]  Nai-Yang Deng,et al.  Support Vector Machines: Optimization Based Theory, Algorithms, and Extensions , 2012 .

[11]  Bernhard Schölkopf,et al.  Support Vector Machine Applications in Computational Biology , 2004 .

[12]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[13]  Michael I. Jordan,et al.  A Robust Minimax Approach to Classification , 2003, J. Mach. Learn. Res..

[14]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[15]  King-Sun Fu,et al.  A Sentence-to-Sentence Clustering Procedure for Pattern Analysis , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[16]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[17]  Michael R. Lyu,et al.  Learning large margin classifiers locally and globally , 2004, ICML.

[18]  Dong Xu,et al.  Robust minimum class variance twin support vector machine classifier , 2011, Neural Computing and Applications.

[19]  Yuqun Zhang,et al.  Structural least square twin support vector machine for classification , 2014, Applied Intelligence.

[20]  Ying-jie Tian,et al.  Improved twin support vector machine , 2013, Science China Mathematics.

[21]  Yong Shi,et al.  Recent advances on support vector machines research , 2012 .

[22]  Jian Yang,et al.  Recursive projection twin support vector machine via within-class variance minimization , 2011, Pattern Recognit..

[23]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[24]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[25]  Yong Shi,et al.  Robust twin support vector machine for pattern classification , 2013, Pattern Recognit..

[26]  Peng Zhang,et al.  SODE: Self-Adaptive One-Dependence Estimators for classification , 2016, Pattern Recognit..

[27]  Yuan-Hai Shao,et al.  Improvements on Twin Support Vector Machines , 2011, IEEE Transactions on Neural Networks.

[28]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[29]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[30]  Qiang Yang,et al.  Structural Regularized Support Vector Machine: A Framework for Structural Large Margin Classifier , 2011, IEEE Transactions on Neural Networks.

[31]  Li Zhang,et al.  Density-induced margin support vector machines , 2011, Pattern Recognit..

[32]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[33]  Mohamed Cheriet,et al.  Model selection for the LS-SVM. Application to handwriting recognition , 2009, Pattern Recognit..

[34]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[35]  Qiang Yang,et al.  Structural Regularized Support Vector Machine , 2011 .

[36]  Daniel S. Yeung,et al.  Structured large margin machines: sensitive to data distributions , 2007, Machine Learning.

[37]  Wentao Mao,et al.  An adaptive support vector regression based on a new sequence of unified orthogonal polynomials , 2013, Pattern Recognit..

[38]  J. H. Ward.  Hierarchical Grouping to Optimize an Objective Function , 1963 .

[39]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[40]  Philip Chan,et al.  Determining the number of clusters/segments in hierarchical clustering/segmentation algorithms , 2004, 16th IEEE International Conference on Tools with Artificial Intelligence.

[41]  Yong Shi,et al.  ν-Nonparallel support vector machine for pattern classification , 2014, Neural Computing and Applications.