Low-rank representation with adaptive dimensionality reduction via manifold optimization for clustering

The dimensionality reduction techniques are often used to reduce data dimensionality for computational efficiency or other purposes in existing low-rank representation (LRR)-based methods. However, the two steps of dimensionality reduction and learning low-rank representation coefficients are implemented in an independent way; thus, the adaptability of representation coefficients to the original data space may not be guaranteed. This paper proposes a novel model, i.e., low-rank representation with adaptive dimensionality reduction (LRRARD) via manifold optimization for clustering, where dimensionality reduction and learning low-rank representation coefficients are integrated into a unified framework. This model introduces a low-dimensional projection matrix to find the projection that best fits the original data space. And the low-dimensional projection matrix and the low-rank representation coefficients interact with each other to simultaneously obtain the best projection matrix and representation coefficients. In addition, a manifold optimization method is employed to obtain the optimal projection matrix, which is an unconstrained optimization method in a constrained search space. The experimental results on several real datasets demonstrate the superiority of our proposed method.

[1]  Zibin Zheng,et al.  A Self-Representation Method with Local Similarity Preserving for Fast Multi-View Outlier Detection , 2023, ACM Trans. Knowl. Discov. Data.

[2]  Xiaocui Peng,et al.  Twin Contrastive Learning for Online Clustering , 2022, International Journal of Computer Vision.

[3]  Yanfeng Sun,et al.  Robust Image Representation via Low Rank Locality Preserving Projection , 2021, ACM Trans. Knowl. Discov. Data.

[4]  Dongxia Chang,et al.  Double low-rank representation with projection distance penalty for clustering , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Jie Chen,et al.  Low-rank representation with adaptive dictionary learning for subspace clustering , 2021, Knowl. Based Syst..

[6]  Xuelong Li,et al.  Weighted bilateral K-means algorithm for fast co-clustering and fast spectral clustering , 2021, Pattern Recognit..

[7]  Zhenwen Ren,et al.  Multiple kernel low-rank representation-based robust multi-view subspace clustering , 2020, Inf. Sci..

[8]  Yiming Wang,et al.  A hierarchical weighted low-rank representation for image clustering and classification , 2020, Pattern Recognit..

[9]  Wenjie Zhu,et al.  Sparse and low-rank regularized deep subspace clustering , 2020, Knowl. Based Syst..

[10]  Minnan Luo,et al.  Self-weighted Robust LDA for Multiclass Classification with Edge Classes , 2020, ACM Trans. Intell. Syst. Technol..

[11]  Dezhong Peng,et al.  Contrastive Clustering , 2021, AAAI.

[12]  Feiping Nie,et al.  Enhanced Balanced Min Cut , 2020, International Journal of Computer Vision.

[13]  Lunke Fei,et al.  Low-Rank Preserving Projection Via Graph Regularized Reconstruction , 2019, IEEE Transactions on Cybernetics.

[14]  Junbin Gao,et al.  Solving Partial Least Squares Regression via Manifold Optimization Approaches , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Yun Fu,et al.  Robust Spectral Ensemble Clustering via Rank Minimization , 2019, ACM Trans. Knowl. Discov. Data.

[16]  Chunwei Tian,et al.  Low-rank representation with adaptive graph regularization , 2018, Neural Networks.

[17]  Jian Yang,et al.  Adaptive weighted nonnegative low-rank representation , 2018, Pattern Recognit..

[18]  Joey Tianyi Zhou,et al.  XAI Beyond Classification: Interpretable Neural Clustering , 2018, J. Mach. Learn. Res..

[19]  Wei-Yun Yau,et al.  Structured AutoEncoders for Subspace Clustering , 2018, IEEE Transactions on Image Processing.

[20]  Nicu Sebe,et al.  Dynamic Affinity Graph Construction for Spectral Clustering Using Multiple Features , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[21]  Feiping Nie,et al.  Rank-Constrained Spectral Clustering With Flexible Embedding , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Hanjiang Lai,et al.  Personalized Age Progression with Bi-Level Aging Dictionary Learning , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Junbin Gao,et al.  Laplacian Regularized Low-Rank Representation and Its Applications , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Junbin Gao,et al.  Partial Sum Minimization of Singular Values Representation on Grassmann Manifolds , 2016, ACM Trans. Knowl. Discov. Data.

[25]  Zhang Yi,et al.  Symmetric low-rank representation for subspace clustering , 2014, Neurocomputing.

[26]  Hiroyuki Sato,et al.  A Dai–Yuan-type Riemannian conjugate gradient method with the weak Wolfe conditions , 2014, Comput. Optim. Appl..

[27]  Hongdong Li,et al.  Efficient dense subspace clustering , 2014, IEEE Winter Conference on Applications of Computer Vision.

[28]  Shuyuan Yang,et al.  Low-rank representation with local constraint for graph construction , 2013, Neurocomputing.

[29]  Nenghai Yu,et al.  Non-negative low rank and sparse graph for semi-supervised learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Zhixun Su,et al.  Fixed-rank representation for unsupervised visual learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[32]  Shuicheng Yan,et al.  Latent Low-Rank Representation for subspace segmentation and feature extraction , 2011, 2011 International Conference on Computer Vision.

[33]  Zhixun Su,et al.  Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation , 2011, NIPS.

[34]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[36]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[37]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[38]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[39]  Ehsan Elhamifar,et al.  Sparse subspace clustering , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[41]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[44]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[45]  Nizar Bouguila,et al.  BLOCK-DBSCAN: Fast clustering for large scale data , 2021, Pattern Recognit..

[46]  Huaijiang Sun,et al.  Robust multiple kernel subspace clustering with block diagonal representation and low-rank consensus kernel , 2021, Knowl. Based Syst..

[47]  H. Oviedo,et al.  A Riemannian Conjugate Gradient Algorithm with Implicit Vector Transport for Optimization on the Stiefel Manifold , 2018 .

[48]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[49]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .