Integrating feature and graph learning with low-rank representation

Abstract We propose a new subspace clustering method that integrates feature and manifold learning while learning a low-rank representation of the data in a single model. This new model seeks a low-rank representation of the data using only the most relevant features in both linear and nonlinear spaces, which helps reveal more accurate data relationships in both linear and nonlinear spaces, because data relationships can be less afflicted by irrelevant features. Moreover, the graph Laplacian is updated according to the learning process, which essentially differs from existing nonlinear subspace clustering methods that require constructing a graph Laplacian as an independent preprocessing step. Thus the learning processes of features and manifold mutually enhance each other and lead to powerful data representations. Extensive experimental results confirm the effectiveness of the proposed method.

[1]  P. Arabie,et al.  Cluster analysis in marketing research , 1994 .

[2]  Deng Cai,et al.  Unsupervised feature selection for multi-cluster data , 2010, KDD.

[3]  Feiping Nie,et al.  Trace Ratio Criterion for Feature Selection , 2008, AAAI.

[4]  René Vidal,et al.  A closed form solution to robust subspace estimation and clustering , 2011, CVPR 2011.

[5]  Zhao Kang,et al.  Subspace Clustering Using Log-determinant Rank Approximation , 2015, KDD.

[6]  René Vidal,et al.  Latent Space Sparse Subspace Clustering , 2013, 2013 IEEE International Conference on Computer Vision.

[7]  Wei-Chien Chang On using Principal Components before Separating a Mixture of Two Multivariate Normal Distributions , 1983 .

[8]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[9]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[10]  Zhao Kang,et al.  Robust Subspace Clustering via Smoothed Rank Approximation , 2015, IEEE Signal Processing Letters.

[11]  Sanja Fidler,et al.  Combining reconstructive and discriminative subspace methods for robust classification and regression by subsampling , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Dong Xu,et al.  Robust Kernel Low-Rank Representation , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[13]  Hans-Peter Kriegel,et al.  Subspace clustering , 2012, WIREs Data Mining Knowl. Discov..

[14]  C. W. Gear,et al.  Multibody Grouping from Motion Images , 1998, International Journal of Computer Vision.

[15]  René Vidal,et al.  A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[17]  Chun Chen,et al.  Sparse fixed-rank representation for robust visual analysis , 2015, Signal Process..

[18]  Peter Benner,et al.  On the ADI method for Sylvester equations , 2009, J. Comput. Appl. Math..

[19]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Jun Yu,et al.  Towards robust subspace recovery via sparsity-constrained latent low-rank representation , 2016, J. Vis. Commun. Image Represent..

[21]  Shuicheng Yan,et al.  Latent Low-Rank Representation for subspace segmentation and feature extraction , 2011, 2011 International Conference on Computer Vision.

[22]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[23]  John Wright,et al.  Segmentation of Multivariate Mixed Data via Lossy Data Coding and Compression , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[25]  Guangliang Chen,et al.  Spectral Curvature Clustering (SCC) , 2009, International Journal of Computer Vision.

[26]  Yong Yu,et al.  Robust Subspace Segmentation by Low-Rank Representation , 2010, ICML.

[27]  Thomas S. Huang,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[28]  René Vidal,et al.  Structured Sparse Subspace Clustering: A unified optimization framework , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Shuicheng Yan,et al.  Robust Subspace Segmentation with Block-Diagonal Prior , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  R. Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications. , 2013, IEEE transactions on pattern analysis and machine intelligence.

[31]  Guillermo Sapiro,et al.  Learning Efficient Sparse and Low Rank Models , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Ming Yang,et al.  Feature Selection Embedded Subspace Clustering , 2016, IEEE Signal Processing Letters.

[33]  René Vidal,et al.  Sparse subspace clustering , 2009, CVPR.

[34]  René Vidal,et al.  Ieee Journal of Selected Topics in Signal Processing, Vol. X, No. X, Month 20xx 1 Latent Space Sparse and Low-rank Subspace Clustering , 2022 .

[35]  Zhao Kang,et al.  Robust Subspace Clustering via Tighter Rank Approximation , 2015, CIKM.

[36]  René Vidal,et al.  Kernel sparse subspace clustering , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[37]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Jim Jing-Yan Wang,et al.  Feature selection and multi-kernel learning for sparse representation on a manifold , 2014, Neural Networks.

[39]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Junbin Gao,et al.  Laplacian Regularized Low-Rank Representation and Its Applications , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Chris H. Q. Ding,et al.  Robust nonnegative matrix factorization using L21-norm , 2011, CIKM '11.

[42]  Xuelong Li,et al.  Multi-view Subspace Clustering , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[43]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[44]  Marc Pollefeys,et al.  A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate , 2006, ECCV.

[45]  Mohamed Nadif,et al.  Simultaneous Semi-NMF and PCA for Clustering , 2015, 2015 IEEE International Conference on Data Mining.

[46]  Junbin Gao,et al.  Kernelized LRR on Grassmann Manifolds for Subspace Clustering , 2016, ArXiv.

[47]  Anna Fabijańska Normalized cuts and watersheds for image segmentation , 2012 .

[48]  Yong Yu,et al.  Unsupervised Object Segmentation with a Hybrid Graph Model (HGM) , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  René Vidal,et al.  Low rank subspace clustering (LRSC) , 2014, Pattern Recognit. Lett..

[50]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[52]  René Vidal,et al.  Motion Segmentation in the Presence of Outlying, Incomplete, or Corrupted Trajectories , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Xiaogang Wang,et al.  A unified framework for subspace face recognition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.