Joint graph optimization and projection learning for dimensionality reduction

Abstract Nowadays, graph-based dimensionality reduction approaches have become more and more popular due to their successful utilization for classification and clustering tasks. In these approaches, how to establish an appropriate graph is critical. To address this issue, a novel graph-based dimensionality reduction framework termed joint graph optimization and projection learning (JGOPL) is proposed in this paper. Compared with existing dimensionality reduction approaches, there are three main advantages of JGOPL. First, through performing the graph optimization and low-dimensional feature learning simultaneously, our proposed approach can accomplish the tasks of graph construction and dimensionality reduction jointly. Second, the l21-norm based distance measurement is adopted in the loss function of our JGOPL so that its robustness to the negative influence caused by the outliers or variations of data can be improved. Third, in order to well exploit and preserve the local structure information of high-dimensional data, a locality constraint is introduced into the proposed JGOPL to discourage a sample from connecting with the distant samples during graph optimization. Extensive classification and clustering experiments are carried out on seven publicly available databases to demonstrate the effectiveness of our approach. At last, the locality constraint and graph optimization strategy proposed in this paper is not only limited to dimensionality reduction, but also can be incorporated into other relevant graph-based tasks (such as spectral clustering).

[1]  Xingyu Wang,et al.  Temporally Constrained Sparse Group Spatial Patterns for Motor Imagery BCI , 2019, IEEE Transactions on Cybernetics.

[2]  Randy C. Paffenroth,et al.  A Nonlinear Dimensionality Reduction Framework Using Smooth Geodesics , 2017, Pattern Recognit..

[3]  Yudong Chen,et al.  Robust jointly sparse embedding for dimensionality reduction , 2018, Neurocomputing.

[4]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[5]  Rosane Minghim,et al.  Content-based text mapping using multi-dimensional projections for exploration of document collections , 2006, Electronic Imaging.

[6]  Hanqing Lu,et al.  Face recognition using kernel scatter-difference-based discriminant analysis , 2006, IEEE Trans. Neural Networks.

[7]  Zhang Yi,et al.  Constructing the L2-Graph for Robust Subspace Learning and Subspace Clustering , 2012, IEEE Transactions on Cybernetics.

[8]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Wai Keung Wong,et al.  Robust Discriminant Regression for Feature Extraction , 2018, IEEE Transactions on Cybernetics.

[10]  K. Kim,et al.  Face recognition using kernel principal component analysis , 2002, IEEE Signal Process. Lett..

[11]  Fei Xiong,et al.  Person Re-Identification Using Kernel-Based Metric Learning Methods , 2014, ECCV.

[12]  Shuicheng Yan,et al.  Learning With $\ell ^{1}$-Graph for Image Analysis , 2010, IEEE Transactions on Image Processing.

[13]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[14]  Zhiping Lin,et al.  An adaptive graph learning method based on dual data representations for clustering , 2018, Pattern Recognit..

[15]  Thomas S. Huang,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[16]  Junbin Gao,et al.  Subspace Clustering via Learning an Adaptive Low-Rank Graph , 2018, IEEE Transactions on Image Processing.

[17]  Patrik O. Hoyer,et al.  Modeling Receptive Fields with Non-Negative Sparse Coding , 2002, Neurocomputing.

[18]  Hui Li,et al.  KCRC-LCD: Discriminative kernel collaborative representation with locality constrained dictionary for visual categorization , 2014, Pattern Recognit..

[19]  Xiaohui Yu,et al.  A Perception-Driven Approach to Supervised Dimensionality Reduction for Visualization , 2018, IEEE Transactions on Visualization and Computer Graphics.

[20]  Sarah Jane Delany k-Nearest Neighbour Classifiers , 2007 .

[21]  Chien-Liang Liu,et al.  Locality-constrained max-margin sparse coding , 2017, Pattern Recognit..

[22]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[23]  Xiaoyang Tan,et al.  Pattern Recognition , 2016, Communications in Computer and Information Science.

[24]  Nojun Kwak,et al.  Independent component analysis by lp-norm optimization , 2018, Pattern Recognit..

[25]  Limei Zhang,et al.  Dimensionality reduction with adaptive graph , 2013, Frontiers of Computer Science.

[26]  Shuicheng Yan,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007 .

[27]  Jian Yang,et al.  Rotational Invariant Dimensionality Reduction Algorithms , 2017, IEEE Transactions on Cybernetics.

[28]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[29]  Shuicheng Yan,et al.  Robust and Efficient Subspace Segmentation via Least Squares Regression , 2012, ECCV.

[30]  Tao Jiang,et al.  Efficient and robust feature extraction by maximum margin criterion , 2003, IEEE Transactions on Neural Networks.

[31]  Chao Bi,et al.  Dimensionality Reduction via Multiple Locality-Constrained Graph Optimization , 2018, IEEE Access.

[32]  Xuelong Li,et al.  Orthogonal self-guided similarity preserving projection for classification and clustering , 2017, Neural Networks.

[33]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[34]  Hwann-Tzong Chen,et al.  Local discriminant embedding and its variants , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[35]  Jian Yang,et al.  A Locality-Constrained and Label Embedding Dictionary Learning Algorithm for Image Classification , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[36]  Andrzej Cichocki,et al.  Linked Component Analysis From Matrices to High-Order Tensors: Applications to Biomedical Data , 2015, Proceedings of the IEEE.

[37]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[38]  Fang Chen,et al.  Dimensionality reduction by LPP-L21 , 2018, IET Comput. Vis..

[39]  Robert Jenssen,et al.  Kernel Entropy Component Analysis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Yu Zhang,et al.  EEG classification using sparse Bayesian extreme learning machine for brain–computer interface , 2018, Neural Computing and Applications.

[41]  Zi Huang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence ℓ2,1-Norm Regularized Discriminative Feature Selection for Unsupervised Learning , 2022 .

[42]  Xingyu Wang,et al.  Fast nonnegative tensor factorization based on accelerated proximal gradient and low-rank approximation , 2016, Neurocomputing.

[43]  Dinggang Shen,et al.  Data-driven graph construction and graph learning: A review , 2018, Neurocomputing.

[44]  I. Jolliffe Principal Component Analysis , 2002 .

[45]  Yang Wang,et al.  Locality constrained Graph Optimization for Dimensionality Reduction , 2017, Neurocomputing.

[46]  Elif Vural,et al.  Nonlinear Supervised Dimensionality Reduction via Smooth Regular Embeddings , 2017, Pattern Recognit..

[47]  Ivica Kopriva,et al.  A Nonlinear Orthogonal Non-Negative Matrix Factorization Approach to Subspace Clustering , 2017, Pattern Recognit..

[48]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[49]  Christopher M. Bishop,et al.  Bayesian PCA , 1998, NIPS.

[50]  Xindong Wu,et al.  10 Challenging Problems in Data Mining Research , 2006, Int. J. Inf. Technol. Decis. Mak..

[51]  Jianzhong Wang,et al.  Ordinal preserving matrix factorization for unsupervised feature selection , 2018, Signal Process. Image Commun..

[52]  Larry S. Davis,et al.  Learning Discriminative Appearance-Based Models Using Partial Least Squares , 2009, 2009 XXII Brazilian Symposium on Computer Graphics and Image Processing.

[53]  Xingyu Wang,et al.  Discriminative Feature Extraction via Multivariate Linear Regression for SSVEP-Based BCI , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[54]  Wai Keung Wong,et al.  Low-Rank Embedding for Robust Image Feature Extraction , 2017, IEEE Transactions on Image Processing.

[55]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[56]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[57]  Limei Zhang,et al.  Graph-optimized locality preserving projections , 2010, Pattern Recognit..

[58]  Jennifer Morgan,et al.  Sparse Generalised Principal Component Analysis , 2018, Pattern Recognit..

[59]  Masashi Sugiyama,et al.  Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis , 2007, J. Mach. Learn. Res..

[60]  David G. Stork,et al.  Pattern Classification , 1973 .

[61]  Bo Yang,et al.  Sample-dependent graph construction with application to dimensionality reduction , 2010, Neurocomputing.

[62]  Limei Zhang,et al.  Graph optimization for dimensionality reduction with sparsity constraints , 2012, Pattern Recognit..

[63]  Zheng Zhang,et al.  Adaptive Locality Preserving Regression , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[64]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[65]  Nicu Sebe,et al.  Flexible Manifold Learning With Optimal Graph for Image and Video Representation , 2018, IEEE Transactions on Image Processing.

[66]  Qi Wang,et al.  Locality constraint distance metric learning for traffic congestion detection , 2018, Pattern Recognit..

[67]  Kevin Chen-Chuan Chang,et al.  A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[68]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.