Improved multi-view GEPSVM via Inter-View Difference Maximization and Intra-view Agreement Minimization

Multiview Generalized Eigenvalue Proximal Support Vector Machine (MvGEPSVM) is an effective method for multiview data classification proposed recently. However, it ignores discriminations between different views and the agreement of the same view. Moreover, there is no robustness guarantee. In this paper, we propose an improved multiview GEPSVM (IMvGEPSVM) method, which adds a multi-view regularization that can connect different views of the same class and simultaneously considers the maximization of the samples from different classes in heterogeneous views for promoting discriminations. This makes the classification more effective. In addition, L1-norm rather than squared L2-norm is employed to calculate the distances from each of the sample points to the hyperplane so as to reduce the effect of outliers in the proposed model. To solve the resulting objective, an efficient iterative algorithm is presented. Theoretically, we conduct the proof of the algorithm's convergence. Experimental results show the effectiveness of the proposed method.

[1]  Gerardo Severino,et al.  On the regularization of generalized eigenvalues classifiers , 2016 .

[2]  Ee-Peng Lim,et al.  Web classification using support vector machine , 2002, WIDM '02.

[3]  Jiashu Zhang,et al.  Linear Discriminant Analysis Based on L1-Norm Maximization , 2013, IEEE Transactions on Image Processing.

[4]  He Yan,et al.  The GEPSVM Classifier Based on L1-Norm Distance Metric , 2016, CCPR.

[5]  Godwin C Ovuworie.  Nonlinear Programming — Theory and Algorithms , 1979 .

[6]  Feiping Nie,et al.  A Non-Greedy Algorithm for L1-Norm LDA , 2017, IEEE Transactions on Image Processing.

[7]  Yuan-Hai Shao,et al.  Robust L1-norm two-dimensional linear discriminant analysis , 2015, Neural Networks.

[8]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[9]  Reshma Khemchandani,et al.  Generalized eigenvalue proximal support vector regressor for the simultaneous learning of a function and its derivatives , 2017, International Journal of Machine Learning and Cybernetics.

[10]  Panos P. Markopoulos,et al.  L1-Norm Principal-Component Analysis of Complex Data , 2017, IEEE Transactions on Signal Processing.

[11]  Yuan-Hai Shao,et al.  Robust L1-norm non-parallel proximal support vector machine , 2016 .

[12]  Yuan Yan Tang,et al.  High-Order Distance-Based Multiview Stochastic Learning in Image Classification , 2014, IEEE Transactions on Cybernetics.

[13]  Yuan-Hai Shao,et al.  Manifold proximal support vector machine for semi-supervised classification , 2013, Applied Intelligence.

[14]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[15]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[16]  Shiliang Sun,et al.  A survey of multi-view machine learning , 2013, Neural Computing and Applications.

[17]  Hao Gao,et al.  A SVM Method Trained by Improved Particle Swarm Optimization for Image Classification , 2014, CCPR.

[18]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Wenjie Hu,et al.  Robust support vector machine with bullet hole image classification , 2002 .

[20]  M. H. Marghny,et al.  Differential Search Algorithm-based Parametric Optimization of Fuzzy Generalized Eigenvalue Proximal Support Vector Machine , 2014, ArXiv.

[21]  Yang Xu.  Localized Proximal Support Vector Machine via Generalized Eigenvalues , 2007 .

[22]  Zhenhua Guo,et al.  Rotation invariant texture classification using LBP variance (LBPV) with global matching , 2010, Pattern Recognit..

[23]  He Yan,et al.  Efficient and robust TWSVM classification via a minimum L1-norm distance metric criterion , 2018, Machine Learning.

[24]  Reshma Khemchandani,et al.  Fuzzy multi-category proximal support vector classification via generalized eigenvalues , 2007, Soft Comput..

[25]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[26]  Guowei Yang,et al.  Multi-View Learning With Robust Generalized Eigenvalue Proximal SVM , 2019, IEEE Access.

[27]  Xuelong Li,et al.  Block-Row Sparse Multiview Multilabel Learning for Image Classification , 2016, IEEE Transactions on Cybernetics.

[28]  Jun Qin,et al.  A SVM face recognition method based on Gabor-featured key points , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[29]  Shiliang Sun,et al.  Multi-view twin support vector machines , 2015, Intell. Data Anal..

[30]  William W. Hsieh,et al.  Machine Learning Methods in the Environmental Sciences: Neural Networks and Kernels , 2009 .

[31]  Yuan-Hai Shao,et al.  MLTSVM: A novel twin support vector machine to multi-label learning , 2016, Pattern Recognit..

[32]  Long Chen,et al.  Manifold Regularized Proximal Support Vector Machine via Generalized Eigenvalue , 2016, Int. J. Comput. Intell. Syst..

[33]  Shiliang Sun,et al.  Multiview Learning With Generalized Eigenvalue Proximal Support Vector Machines , 2019, IEEE Transactions on Cybernetics.

[34]  Shiguang Shan,et al.  Multi-View Discriminant Analysis , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Xuesong Lu,et al.  Fisher Discriminant Analysis With L1-Norm , 2014, IEEE Transactions on Cybernetics.

[36]  Jian Yang,et al.  L1-Norm Distance Linear Discriminant Analysis Based on an Effective Iterative Algorithm , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[37]  Xiaogang Wang,et al.  Face Photo-Sketch Synthesis and Recognition , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Yong Luo,et al.  Large Margin Multi-Modal Multi-Task Feature Extraction for Image Classification , 2019, IEEE Transactions on Image Processing.

[39]  Ling Shao,et al.  Action Recognition Using 3D Histograms of Texture and A Multi-Class Boosting Classifier , 2017, IEEE Transactions on Image Processing.

[40]  Jingjing Tang,et al.  Multiview Privileged Support Vector Machines , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[41]  Tao Liu,et al.  Deep convolutional neural network training enrichment using multi-view object-based analysis of Unmanned Aerial systems imagery for wetlands classification , 2018 .

[42]  Mokhtar S. Bazaraa,et al.  Nonlinear Programming: Theory and Algorithms , 1993 .

[43]  Shiliang Sun,et al.  Multiview Uncorrelated Discriminant Analysis , 2016, IEEE Transactions on Cybernetics.

[44]  Yuan-Hai Shao,et al.  Improved Generalized Eigenvalue Proximal Support Vector Machine , 2013, IEEE Signal Processing Letters.

[45]  Nai-Yang Deng,et al.  Support Vector Machines: Optimization Based Theory, Algorithms, and Extensions , 2012 .

[46]  Yuan-Hai Shao,et al.  An efficient weighted Lagrangian twin support vector machine for imbalanced data classification , 2014, Pattern Recognit..

[47]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.