Adaptive sparse graph learning based dimensionality reduction for classification

Abstract To preserve the sparsity structure in dimensionality reduction, sparsity preserving projection (SPP) is widely used in many fields of classification, which has the advantages of noise robustness and data adaptivity compared with other graph based method. However, the sparsity parameter of SPP is fixed for all samples without any adjustment. In this paper, an improved SPP method is proposed, which has an adaptive parameter adjustment strategy during sparse graph construction. With this adjustment strategy, the sparsity parameter of each sample is adjusted adaptively according to the relationship of those samples with nonzero sparse representation coefficients, by which the discriminant information of graph is enhanced. With the same expectation, similarity information both in original space and projection space is applied for sparse representation as guidance information. Besides, a new measurement is introduced to control the influence of each sample’s local structure on projection learning, by which more correct discriminant information should be preserved in the projection space. With the contributions of above strategies, the low-dimensional space with high discriminant ability is found, which is more beneficial for classification. Experimental results on three datasets demonstrate that the proposed approach can achieve better classification performance over some available state-of-the-art approaches.

[1]  Jing Wang,et al.  Robust Face Recognition via Adaptive Sparse Representation , 2014, IEEE Transactions on Cybernetics.

[2]  Zhaohui Wu,et al.  Accelerometer-Based Gait Recognition by Sparse Representation of Signature Points With Clusters , 2015, IEEE Transactions on Cybernetics.

[3]  Tanaya Guha,et al.  Learning Sparse Representations for Human Action Recognition , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[5]  Marco La Cascia,et al.  3D skeleton-based human action classification: A survey , 2016, Pattern Recognit..

[6]  Qian Du,et al.  Sparse Graph-Based Discriminant Analysis for Hyperspectral Imagery , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Liangpei Zhang,et al.  Spectral–Spatial Sparse Subspace Clustering for Hyperspectral Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Anil K. Jain,et al.  Feature extraction methods for character recognition-A survey , 1996, Pattern Recognit..

[9]  Fadi Dornaika,et al.  Exponential Local Discriminant Embedding and Its Application to Face Recognition , 2013, IEEE Transactions on Cybernetics.

[10]  René Vidal,et al.  Sparse Manifold Clustering and Embedding , 2011, NIPS.

[11]  Shuicheng Yan,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007 .

[12]  Trevor J. Hastie,et al.  Sparse Discriminant Analysis , 2011, Technometrics.

[13]  Hwann-Tzong Chen,et al.  Local discriminant embedding and its variants , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  Liangpei Zhang,et al.  A Nonlocal Weighted Joint Sparse Representation Classification Method for Hyperspectral Imagery , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[15]  Jiamin Liu,et al.  Local Geometric Structure Feature for Dimensionality Reduction of Hyperspectral Imagery , 2017, Remote. Sens..

[16]  Anil M. Cheriyadat,et al.  Unsupervised Feature Learning for Aerial Scene Classification , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[17]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[18]  Tommy W. S. Chow,et al.  M-Isomap: Orthogonal Constrained Marginal Isomap for Nonlinear Dimensionality Reduction , 2013, IEEE Transactions on Cybernetics.

[19]  Qi Wang,et al.  Hyperspectral Image Classification via Multitask Joint Sparse Representation and Stepwise MRF Optimization , 2016, IEEE Transactions on Cybernetics.

[20]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[22]  H. Hotelling.  Analysis of a complex of statistical variables into principal components. , 1933 .

[23]  Fang Liu,et al.  Semi-supervised double sparse graphs based discriminant analysis for dimensionality reduction , 2017, Pattern Recognit..

[24]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[25]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[26]  Shuicheng Yan,et al.  Learning With $\ell ^{1}$-Graph for Image Analysis , 2010, IEEE Transactions on Image Processing.

[27]  Mei Yang,et al.  Dimensionality Reduction of Hyperspectral Images With Sparse Discriminant Embedding , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Fernando De la Torre,et al.  A Least-Squares Framework for Component Analysis , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[30]  R. Tibshirani,et al.  Sparse Principal Component Analysis , 2006 .

[31]  Jianhua Z. Huang,et al.  Sparse principal component analysis via regularized low rank matrix approximation , 2008 .

[32]  Jian Yang,et al.  Generalized K-L transform based combined feature extraction , 2002, Pattern Recognit..

[33]  Fang Liu,et al.  Dimensionality reduction for hyperspectral image classification based on multiview graphs ensemble , 2016 .

[34]  Christopher K. I. Williams.  On a Connection between Kernel PCA and Metric Multidimensional Scaling , 2004, Machine Learning.

[35]  Konstantinos N. Plataniotis,et al.  An efficient kernel discriminant analysis method , 2005, Pattern Recognit..

[36]  R. Fisher.  THE STATISTICAL UTILIZATION OF MULTIPLE MEASUREMENTS , 1938 .

[37]  Wei Jia,et al.  Discriminant sparse neighborhood preserving embedding for face recognition , 2012, Pattern Recognit..

[38]  Zhouyu Fu,et al.  A Survey of Audio-Based Music Classification and Annotation , 2011, IEEE Transactions on Multimedia.

[39]  Douglas L. Medin,et al.  Context theory of classification learning. , 1978 .

[40]  Jiawei Han,et al.  Spectral Regression: A Unified Approach for Sparse Subspace Learning , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[41]  Bo Yang,et al.  Linear dimensionality reduction based on Hybrid structure preserving projections , 2016, Neurocomputing.

[42]  Masashi Sugiyama,et al.  Local Fisher discriminant analysis for supervised dimensionality reduction , 2006, ICML.

[43]  Lijuan Cao,et al.  A comparison of PCA, KPCA and ICA for dimensionality reduction in support vector machine , 2003, Neurocomputing.

[44]  H. Zha,et al.  Principal manifolds and nonlinear dimensionality reduction via tangent space alignment , 2004, SIAM J. Sci. Comput..

[45]  Jian Yang,et al.  Sparse Representation Classifier Steered Discriminative Projection With Applications to Face Recognition , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[46]  Fang Liu,et al.  Dimensionality Reduction of Hyperspectral Imagery Using Sparse Graph Learning , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[47]  Xiaoyang Tan,et al.  Pattern Recognition , 2016, Communications in Computer and Information Science.

[48]  Amnon Shashua,et al.  Nonnegative Sparse PCA , 2006, NIPS.