A novel low-rank hypergraph feature selection for multi-view classification

In order to select informative features from a high-dimensional multi-view dataset, we have proposed a feature selection method that simultaneously embedding the low-rank constraint, sparse representation, global and local structure learning into a unified framework. Firstly, we utilize the conventional regression function (i.e. the least square loss function) to form a novel regression framework by introducing a low-rank constraint and a relaxation term. And then we employ an l21-norm regularization term to filter out the redundant and irrelative features. Furthermore, we utilize a hypergraph based regularization term rather than the simple graph to construct a Laplacian matrix that will be used in enhancing the inherent association of data. Besides, we proposed a novel optimization algorithm to solve the objective function. Finally, we feed the reduced data got by the proposed feature selection method into Support Vector Machines (SVM) in term of classification accuracy. The experimental results showed that the proposed method achieved the best classification performance, compared with the state-of-the-art feature selection methods on real multi-view dataset.

[1]  P. Corsini Hypergraphs and hypergroups , 1996 .

[2]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Chengqi Zhang,et al.  Cost-Sensitive Imputing Missing Values with Ordering , 2007, AAAI.

[4]  Guang-Hong Yang,et al.  Robust Distributed Fault Estimation for a Network of Dynamical Systems , 2018, IEEE Transactions on Control of Network Systems.

[5]  Chengqi Zhang,et al.  Semi-parametric optimization for missing data imputation , 2007, Applied Intelligence.

[6]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  M. Wegkamp,et al.  Optimal selection of reduced rank estimators of high-dimensional matrices , 2010, 1004.2995.

[8]  Jing Wang,et al.  Robust Face Recognition via Adaptive Sparse Representation , 2014, IEEE Transactions on Cybernetics.

[9]  Liqun Qi,et al.  The Laplacian of a uniform hypergraph , 2015, J. Comb. Optim..

[10]  Meng Wang,et al.  Adaptive Hypergraph Learning and its Application in Image Classification , 2012, IEEE Transactions on Image Processing.

[11]  Witold Pedrycz,et al.  Subspace learning for unsupervised feature selection via matrix factorization , 2015, Pattern Recognit..

[12]  Johan A. K. Suykens,et al.  Low rank updated LS-SVM classifiers for fast variable selection , 2008, Neural Networks.

[13]  Lei Wang,et al.  Global and Local Structure Preservation for Feature Selection , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Dae-Won Kim,et al.  Accelerating Multi-Label Feature Selection Based on Low-Rank Approximation , 2016, IEICE Trans. Inf. Syst..

[15]  Shichao Zhang,et al.  Feature selection by combining subspace learning with sparse representation , 2017, Multimedia Systems.

[16]  Dinggang Shen,et al.  A Novel Multi-relation Regularization Method for Regression and Classification in AD Diagnosis , 2014, MICCAI.

[17]  Miklós Simonovits,et al.  Supersaturated graphs and hypergraphs , 1983, Comb..

[18]  Zihan Zhou,et al.  Demo: Robust face recognition via sparse representation , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[19]  G. Reinsel,et al.  Multivariate Reduced-Rank Regression: Theory and Applications , 1998 .

[20]  Dinggang Shen,et al.  A novel relational regularization feature selection method for joint regression and classification in AD diagnosis , 2017, Medical Image Anal..

[21]  Yihong Gong,et al.  Unsupervised Image Categorization by Hypergraph Partition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Shichao Zhang,et al.  Robust Joint Graph Sparse Coding for Unsupervised Spectral Feature Selection , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Zili Zhang,et al.  Missing Value Estimation for Mixed-Attribute Data Sets , 2011, IEEE Transactions on Knowledge and Data Engineering.

[24]  Xuelong Li,et al.  Block-Row Sparse Multiview Multilabel Learning for Image Classification , 2016, IEEE Transactions on Cybernetics.

[25]  Fuqiang Chen,et al.  Effective feature selection using feature vector graph for classification , 2015, Neurocomputing.

[26]  Zi Huang,et al.  A Sparse Embedding and Least Variance Encoding Approach to Hashing , 2014, IEEE Transactions on Image Processing.

[27]  Rama Chellappa,et al.  Multiple Kernel Learning for Sparse Representation-Based Classification , 2014, IEEE Transactions on Image Processing.

[28]  Simon Lucey,et al.  Complex Non-rigid Motion 3D Reconstruction by Union of Subspaces , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Shuzhi Sam Ge,et al.  Constrained Multilegged Robot System Modeling and Fuzzy Control With Uncertain Kinematics and Dynamics Incorporating Foot Force Optimization , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[30]  Jieping Ye,et al.  Optimal exact least squares rank minimization , 2012, KDD.

[31]  Nilanjan Ray,et al.  Joint Feature Selection with Low-rank Dictionary Learning , 2015, BMVC.

[32]  Xiaofeng Zhu,et al.  Video-to-Shot Tag Propagation by Graph Sparse Group Lasso , 2013, IEEE Transactions on Multimedia.

[33]  Shuicheng Yan,et al.  Latent Low-Rank Representation for subspace segmentation and feature extraction , 2011, 2011 International Conference on Computer Vision.

[34]  Thomas E. Nichols,et al.  Discovering genetic associations with high-dimensional neuroimaging phenotypes: A sparse reduced-rank regression approach , 2010, NeuroImage.

[35]  Shiliang Sun,et al.  A survey of multi-view machine learning , 2013, Neural Computing and Applications.

[36]  Zi Huang,et al.  Self-taught dimensionality reduction on the high-dimensional small-sized data , 2013, Pattern Recognit..

[37]  Jiawei Han,et al.  Joint Feature Selection and Subspace Learning , 2011, IJCAI.

[38]  C. Ding,et al.  On the equivalent of low-rank linear regressions and linear discriminant analysis based regressions , 2013, KDD.

[39]  Thomas S. Huang,et al.  Pose-robust face recognition via sparse representation , 2013, Pattern Recognit..

[40]  Michael B. Cohen,et al.  Dimensionality Reduction for k-Means Clustering and Low Rank Approximation , 2014, STOC.

[41]  Dinggang Shen,et al.  Subspace Regularized Sparse Multitask Learning for Multiclass Neurodegenerative Disease Identification , 2016, IEEE Transactions on Biomedical Engineering.

[42]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.