Nonnegative self-representation with a fixed rank constraint for subspace clustering

Abstract A number of approaches to graph-based subspace clustering, which assumes that the clustered data points were drawn from an unknown union of multiple subspaces, have been proposed in recent years. Despite their successes in computer vision and data mining, most neglect to simultaneously consider global and local information, which may improve clustering performance. On the other hand, the number of connected components reflected by the learned affinity matrix is commonly inconsistent with the true number of clusters. To this end, we propose an adaptive affinity matrix learning method, nonnegative self-representation with a fixed rank constraint (NSFRC), in which the nonnegative self-representation and an adaptive distance regularization jointly uncover the intrinsic structure of data. In particular, a fixed rank constraint as a prior is imposed on the Laplacian matrix associated with the data representation coefficients to urge the true number of clusters to exactly equal the number of connected components in the learned affinity matrix. Also, we derive an efficient iterative algorithm based on an augmented Lagrangian multiplier to optimize NSFRC. Extensive experiments conducted on real-world benchmark datasets demonstrate the superior performance of the proposed method over some state-of-the-art approaches.

[1]  R. Tibshirani,et al.  Sparse Principal Component Analysis , 2006 .

[2]  K. Fan On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations: II. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Xiao Zhang,et al.  Finding Celebrities in Billions of Web Images , 2012, IEEE Transactions on Multimedia.

[4]  Yulong Wang,et al.  Graph-Regularized Low-Rank Representation for Destriping of Hyperspectral Images , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Shichao Zhang,et al.  Low-Rank Sparse Subspace for Spectral Clustering , 2019, IEEE Transactions on Knowledge and Data Engineering.

[7]  Ran He,et al.  Nonnegative sparse coding for discriminative semi-supervised learning , 2011, CVPR 2011.

[8]  Yong Wang,et al.  Low-Rank Matrix Factorization With Adaptive Graph Regularizer , 2016, IEEE Transactions on Image Processing.

[9]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[10]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[11]  Yue Gao,et al.  Correntropy-Induced Robust Low-Rank Hypergraph , 2019, IEEE Transactions on Image Processing.

[12]  Yuantao Gu,et al.  Active Orthogonal Matching Pursuit for Sparse Subspace Clustering , 2018, IEEE Signal Processing Letters.

[13]  Jianping Fan,et al.  A generalized least-squares approach regularized with graph embedding for dimensionality reduction , 2020, Pattern Recognit..

[14]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[15]  Ronen Basri,et al.  Lambertian Reflectance and Linear Subspaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Ramón Díaz-Uriarte,et al.  Gene selection and classification of microarray data using random forest , 2006, BMC Bioinformatics.

[17]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[18]  Thomas S. Huang,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[19]  Junbin Gao,et al.  Subspace Clustering via Learning an Adaptive Low-Rank Graph , 2018, IEEE Transactions on Image Processing.

[20]  Chunwei Tian,et al.  Low-rank representation with adaptive graph regularization , 2018, Neural Networks.

[21]  Shuicheng Yan,et al.  Robust and Efficient Subspace Segmentation via Least Squares Regression , 2012, ECCV.

[22]  David J. Kriegman,et al.  From few to many: generative models for recognition under variable pose and illumination , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[23]  René Vidal,et al.  Subspace Clustering , 2011, IEEE Signal Processing Magazine.

[24]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[25]  Matthias Hein,et al.  Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization , 2012, 1205.0953.

[26]  Jian Yang,et al.  LRR for Subspace Segmentation via Tractable Schatten- $p$ Norm Minimization and Factorization , 2019, IEEE Transactions on Cybernetics.

[27]  Zongben Xu,et al.  Enhancing Low-Rank Subspace Clustering by Manifold Regularization , 2014, IEEE Transactions on Image Processing.

[28]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[29]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[30]  Ivor W. Tsang,et al.  Spectral Embedded Clustering: A Framework for In-Sample and Out-of-Sample Spectral Clustering , 2011, IEEE Transactions on Neural Networks.

[31]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Ming-Hsuan Yang,et al.  Subspace Clustering via Good Neighbors , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Junbin Gao,et al.  Laplacian Regularized Low-Rank Representation and Its Applications , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[35]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[37]  Claudio Turchetti,et al.  A manifold learning approach to dimensionality reduction for modeling data , 2019, Inf. Sci..

[38]  G. J. Borse,et al.  Numerical Methods with MATLAB: A Resource for Scientists and Engineers , 1996 .

[39]  Svetha Venkatesh,et al.  Improved subspace clustering via exploitation of spatial constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Feiping Nie,et al.  Clustering and projected clustering with adaptive neighbors , 2014, KDD.

[41]  Weilin Huang,et al.  Adaptive nonlinear manifolds and their applications to pattern recognition , 2010, Inf. Sci..

[42]  David Zhang,et al.  Sparse, Collaborative, or Nonnegative Representation: Which Helps Pattern Classification? , 2018, Pattern Recognition.