Low-Rank Sparse Subspace for Spectral Clustering

Traditional graph clustering methods consist of two sequential steps, i.e., constructing an affinity matrix from the original data and then performing spectral clustering on the resulting affinity matrix. This two-step strategy achieves optimal solution for each step separately, but cannot guarantee that it will obtain the globally optimal clustering results. Moreover, the affinity matrix directly learned from the original data will seriously affect the clustering performance, since high-dimensional data are usually noisy and may contain redundancy. To address the above issues, this paper proposes a Low-rank Sparse Subspace (LSS) clustering method via dynamically learning the affinity matrix from low-dimensional space of the original data. Specifically, we learn a transformation matrix to project the original data to their low-dimensional space, by conducting feature selection and subspace learning in the sample self-representation framework. Then, we utilize the rank constraint and the affinity matrix directly obtained from the original data to construct a dynamic and intrinsic affinity matrix. Moreover, each of these three matrices is updated iteratively while fixing the other two. In this way, the affinity matrix learned from the low-dimensional space is the final clustering results. Extensive experiments are conducted on both synthetic and real datasets to show that our proposed LSS method outperforms the state-of-the-art clustering methods.

[1]  Fang Chen,et al.  Spectral clustering of high-dimensional data via Nonnegative Matrix Factorization , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[2]  Ting Luo,et al.  An Improved DBSCAN Algorithm to Detect Stops in Individual Trajectories , 2017, ISPRS Int. J. Geo Inf..

[3]  Zi Huang,et al.  Self-taught dimensionality reduction on the high-dimensional small-sized data , 2013, Pattern Recognit..

[4]  Wotao Yin,et al.  A feasible method for optimization with orthogonality constraints , 2013, Math. Program..

[5]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[6]  Xiaofeng Zhu,et al.  Local and Global Structure Preservation for Robust Unsupervised Spectral Feature Selection , 2018, IEEE Transactions on Knowledge and Data Engineering.

[7]  Longbing Cao,et al.  A novel graph-based k-means for nonlinear manifold clustering and representative selection , 2014, Neurocomputing.

[8]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[9]  Jörg Sander Density-Based Clustering , 2017, Encyclopedia of Machine Learning and Data Mining.

[10]  Feiping Nie,et al.  Clustering and projected clustering with adaptive neighbors , 2014, KDD.

[11]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[12]  Kristin J. Dana,et al.  Modified balanced iterative reducing and clustering using hierarchies (m-BIRCH) for visual clustering , 2016, Pattern Analysis and Applications.

[13]  Wenjun Zhou,et al.  Spectral clustering of high-dimensional data exploiting sparse representation vectors , 2014, Neurocomputing.

[14]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[15]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Jitendra Malik,et al.  Color- and texture-based image segmentation using EM and its application to content-based image retrieval , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[17]  Zi Huang,et al.  Sparse hashing for fast multimedia search , 2013, TOIS.

[18]  Christoph Schnörr,et al.  Spectral clustering of linear subspaces for motion segmentation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[19]  Xiaofeng Zhu,et al.  Dynamic graph learning for spectral feature selection , 2018, Multimedia Tools and Applications.

[20]  K. Fan On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations: II. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[21]  Xiaofeng Zhu,et al.  Unsupervised feature selection by self-paced learning regularization , 2020, Pattern Recognit. Lett..

[22]  B. Mohar THE LAPLACIAN SPECTRUM OF GRAPHS y , 1991 .

[23]  Jonathan T. Barron,et al.  Multiscale Combinatorial Grouping , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Dilip B. Kotak,et al.  GRIDBSCAN: GRId Density-Based Spatial Clustering of Applications with Noise , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[25]  Samuel Kaski,et al.  Informative Discriminant Analysis , 2003, ICML.

[26]  Timo Kohlberger,et al.  Multi-stage Learning for Robust Lung Segmentation in Challenging CT Volumes , 2011, MICCAI.

[27]  D. B. Graham,et al.  Characterising Virtual Eigensignatures for General Purpose Face Recognition , 1998 .

[28]  Xiaofeng Zhu,et al.  Graph self-representation method for unsupervised feature selection , 2017, Neurocomputing.

[29]  Zi Huang,et al.  Dimensionality reduction by Mixed Kernel Canonical Correlation Analysis , 2012, Pattern Recognition.

[30]  Hai Jin,et al.  Color Image Segmentation Based on Mean Shift and Normalized Cuts , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[31]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Jianjiang Feng,et al.  Smooth Representation Clustering , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Dinggang Shen,et al.  A novel relational regularization feature selection method for joint regression and classification in AD diagnosis , 2017, Medical Image Anal..

[34]  Xiaofeng Zhu,et al.  Efficient kNN Classification With Different Numbers of Nearest Neighbors , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Xuelong Li,et al.  Graph PCA Hashing for Similarity Search , 2017, IEEE Transactions on Multimedia.

[36]  Andrew B. Kahng,et al.  New spectral methods for ratio cut partitioning and clustering , 1991, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[37]  R. Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications. , 2013, IEEE transactions on pattern analysis and machine intelligence.

[38]  Jong Won Yun,et al.  Time-course microarrays reveal early activation of the immune transcriptome and adipokine dysregulation leads to fibrosis in visceral adipose depots during diet-induced obesity , 2012, BMC Genomics.

[39]  Xiaofeng Zhu,et al.  Unsupervised feature selection via local structure learning and sparse learning , 2017, Multimedia Tools and Applications.

[40]  Cristian Sminchisescu,et al.  CPMC: Automatic Object Segmentation Using Constrained Parametric Min-Cuts , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Xuelong Li,et al.  Block-Row Sparse Multiview Multilabel Learning for Image Classification , 2016, IEEE Transactions on Cybernetics.

[42]  Xuelong Li,et al.  Learning k for kNN Classification , 2017, ACM Trans. Intell. Syst. Technol..

[43]  Renfa Li,et al.  EECR: Energy Efficient Clustering Using Representatives for Wireless Sensor Networks , 2015 .

[44]  K. Fan On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations I. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[45]  Claire Cardie,et al.  Constrained K-means Clustering with Background Knowledge , 2001, ICML.

[46]  David Zhang,et al.  A survey of graph theoretical approaches to image segmentation , 2013, Pattern Recognit..

[47]  Klemens Böhm,et al.  Improving Accuracy and Robustness of Self-Tuning Histograms by Subspace Clustering , 2015, IEEE Transactions on Knowledge and Data Engineering.

[48]  Feiping Nie,et al.  A New Simplex Sparse Learning Model to Measure Data Similarity for Clustering , 2015, IJCAI.

[49]  Zi Huang,et al.  A Sparse Embedding and Least Variance Encoding Approach to Hashing , 2014, IEEE Transactions on Image Processing.

[50]  Feiping Nie,et al.  The Constrained Laplacian Rank Algorithm for Graph-Based Clustering , 2016, AAAI.

[51]  Ashish Sharma,et al.  An Enhanced Density Based Spatial Clustering of Applications with Noise , 2009, 2009 IEEE International Advance Computing Conference.

[52]  Shuicheng Yan,et al.  Robust and Efficient Subspace Segmentation via Least Squares Regression , 2012, ECCV.

[53]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[54]  Hans-Peter Kriegel,et al.  Density‐based clustering , 2011, WIREs Data Mining Knowl. Discov..

[55]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[56]  I. Daubechies,et al.  Iteratively reweighted least squares minimization for sparse recovery , 2008, 0807.0575.