Sparse subspace clustering for data with missing entries and high-rank matrix completion

Many methods have recently been proposed for subspace clustering, but they are often unable to handle incomplete data because of missing entries. Using matrix completion methods to recover missing entries is a common way to solve the problem. Conventional matrix completion methods require that the matrix should be of low-rank intrinsically, but most matrices are of high-rank or even full-rank in practice, especially when the number of subspaces is large. In this paper, a new method called Sparse Representation with Missing Entries and Matrix Completion is proposed to solve the problems of incomplete-data subspace clustering and high-rank matrix completion. The proposed algorithm alternately computes the matrix of sparse representation coefficients and recovers the missing entries of a data matrix. The proposed algorithm recovers missing entries through minimizing the representation coefficients, representation errors, and matrix rank. Thorough experimental study and comparative analysis based on synthetic data and natural images were conducted. The presented results demonstrate that the proposed algorithm is more effective in subspace clustering and matrix completion compared with other existing methods.

[1]  Yong Peng,et al.  Enhanced low-rank representation via sparse manifold adaption for semi-supervised learning , 2015, Neural Networks.

[2]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[3]  Yi Yang,et al.  Discriminative Nonnegative Spectral Clustering with Out-of-Sample Extension , 2013, IEEE Transactions on Knowledge and Data Engineering.

[4]  I. Jolliffe.  Principal Component Analysis , 2002 .

[5]  Allen Y. Yang,et al.  Unsupervised segmentation of natural images via lossy data compression , 2008, Comput. Vis. Image Underst..

[6]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[7]  C. W. Gear,et al.  Multibody Grouping from Motion Images , 1998, International Journal of Computer Vision.

[8]  Alexandre Bernardino,et al.  Matrix Completion for Weakly-Supervised Multi-Label Image Classification , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[10]  Daniel P. Robinson,et al.  Sparse Subspace Clustering with Missing Entries , 2015, ICML.

[11]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[12]  Kathrin Klamroth,et al.  Biconvex sets and optimization with biconvex functions: a survey and extensions , 2007, Math. Methods Oper. Res..

[13]  Wilfred Ng,et al.  Expert Finding for Question Answering via Graph Regularized Matrix Completion , 2015, IEEE Transactions on Knowledge and Data Engineering.

[14]  Victor Vianu,et al.  Invited articles section foreword , 2010, JACM.

[15]  Ke-Lin Du,et al.  Clustering: A neural network approach , 2010, Neural Networks.

[16]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[17]  Huan Liu,et al.  Subspace clustering for high dimensional data: a review , 2004, SKDD.

[18]  Prateek Jain,et al.  Universal Matrix Completion , 2014, ICML.

[19]  Robert D. Nowak,et al.  High-Rank Matrix Completion and Subspace Clustering with Missing Data , 2011, ArXiv.

[20]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[22]  René Vidal,et al.  Multiframe Motion Segmentation with Missing Data Using PowerFactorization and GPCA , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[23]  Inderjit S. Dhillon,et al.  Fast Multiplier Methods to Optimize Non-exhaustive, Overlapping Clustering , 2016, SDM.

[24]  Ehsan Elhamifar,et al.  High-Rank Matrix Completion and Clustering under Self-Expressive Models , 2016, NIPS.

[25]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[26]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[27]  Chun Chen,et al.  EMR: A Scalable Graph-Based Ranking Model for Content-Based Image Retrieval , 2015, IEEE Transactions on Knowledge and Data Engineering.

[28]  A. Tsybakov,et al.  Robust matrix completion , 2014, Probability Theory and Related Fields.

[29]  Yong Luo,et al.  Multiview matrix completion for multilabel image classification. , 2015, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[30]  Arthur Zimek,et al.  A survey on enhanced subspace clustering , 2013, Data Mining and Knowledge Discovery.

[31]  David B. Dunson,et al.  Subspace segmentation by dense block and sparse representation , 2016, Neural Networks.

[32]  Shuicheng Yan,et al.  Latent Low-Rank Representation for subspace segmentation and feature extraction , 2011, 2011 International Conference on Computer Vision.

[33]  Kai Zhang,et al.  Extreme learning machine and adaptive sparse representation for image classification , 2016, Neural Networks.

[34]  Harrison H. Zhou,et al.  Structured matrix estimation and completion , 2017, Bernoulli.

[35]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[36]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[37]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  R. Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications. , 2013, IEEE transactions on pattern analysis and machine intelligence.

[39]  Hans-Peter Kriegel,et al.  Clustering high-dimensional data: A survey on subspace clustering, pattern-based clustering, and correlation clustering , 2009, TKDD.

[40]  Jicong Fan,et al.  Fault detection and diagnosis of non-linear non-Gaussian dynamic processes using kernel dynamic independent component analysis , 2014, Inf. Sci..

[41]  Hans-Peter Kriegel,et al.  Subspace clustering , 2012, WIREs Data Mining Knowl. Discov..

[42]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[44]  René Vidal,et al.  Ieee Journal of Selected Topics in Signal Processing, Vol. X, No. X, Month 20xx 1 Latent Space Sparse and Low-rank Subspace Clustering , 2022 .

[45]  René Vidal,et al.  Segmenting Motions of Different Types by Unsupervised Manifold Clustering , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Gilad Lerman,et al.  Hybrid Linear Modeling via Local Best-Fit Flats , 2010, International Journal of Computer Vision.

[47]  Zhongfei Zhang,et al.  Context-Aware Hypergraph Construction for Robust Spectral Clustering , 2014, 1401.0764.

[48]  Marc Pollefeys,et al.  A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate , 2006, ECCV.

[49]  Mahdi Soltanolkotabi,et al.  From subspace clustering to full-rank matrix completion , .

[50]  Fang Liu,et al.  An efficient matrix bi-factorization alternative optimization method for low-rank matrix recovery and completion , 2013, Neural Networks.

[51]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[52]  Dong Xu,et al.  Robust Kernel Low-Rank Representation , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[53]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[54]  Xindong Wu,et al.  Data mining with big data , 2014, IEEE Transactions on Knowledge and Data Engineering.

[55]  Zhixun Su,et al.  Learning Markov random walks for robust subspace clustering and estimation , 2014, Neural Networks.

[56]  Shuyuan Yang,et al.  Classification and saliency detection by semi-supervised low-rank representation , 2016, Pattern Recognit..

[57]  Heiko Hoffmann,et al.  Kernel PCA for novelty detection , 2007, Pattern Recognit..

[58]  Nabil H. Mustafa,et al.  k-means projective clustering , 2004, PODS.

[59]  Volker Blanz,et al.  Component-Based Face Recognition with 3D Morphable Models , 2004, CVPR Workshops.