Fast subspace segmentation via Random Sample Probing

Abstract Subspace segmentation is to group a given set of n data points into multiple clusters, with each cluster corresponding to a subspace. Prevalent methods such as Sparse Subspace Clustering (SSC) and Low-Rank Representation (LRR) are effective in terms of segmentation accuracy, but computationally inefficient while applying to gigantic datasets where n is very large as they possess a complexity of O(n3). In this paper, we propose an iterative method called Random Sample Probing (RANSP). In each iteration, RANSP finds the members of one subspace by randomly choosing a data point (called “seed”) at first, and then using Ridge Regression (RR) to retrieve the other points that belong to the same subspace as the seed. Such a procedure is repeated until all points have been classified. RANSP has a computational complexity of O(n) and can therefore handle large-scale datasets. Experiments on synthetic and real datasets confirm the effectiveness and efficiency of RANSP.

[1]  Shuicheng Yan,et al.  Efficient Subspace Segmentation via Quadratic Programming , 2011, AAAI.

[2]  Shiqian Ma,et al.  Barzilai-Borwein Step Size for Stochastic Gradient Descent , 2016, NIPS.

[3]  Qingshan Liu,et al.  A Deterministic Analysis for LRR , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Shuicheng Yan,et al.  Robust and Efficient Subspace Segmentation via Least Squares Regression , 2012, ECCV.

[5]  Huan Xu,et al.  Provable Subspace Clustering: When LRR Meets SSC , 2013, IEEE Transactions on Information Theory.

[6]  John Wright,et al.  Segmentation of Multivariate Mixed Data via Lossy Data Coding and Compression , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  René Vidal,et al.  A closed form solution to robust subspace estimation and clustering , 2011, CVPR 2011.

[8]  Jun Wang,et al.  LRSR: Low-Rank-Sparse representation for subspace clustering , 2016, Neurocomputing.

[9]  Tong Zhang,et al.  Accelerating Stochastic Gradient Descent using Predictive Variance Reduction , 2013, NIPS.

[10]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Jiawei Han,et al.  Document clustering using locality preserving indexing , 2005, IEEE Transactions on Knowledge and Data Engineering.

[12]  Zhang Yi,et al.  Scalable Sparse Subspace Clustering , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  René Vidal,et al.  Sparse subspace clustering , 2009, CVPR.

[14]  Shuicheng Yan,et al.  Latent Low-Rank Representation for subspace segmentation and feature extraction , 2011, 2011 International Conference on Computer Vision.

[15]  George Atia,et al.  Innovation Pursuit: A New Approach to Subspace Clustering , 2015, IEEE Transactions on Signal Processing.

[16]  Bing Li,et al.  ON PRINCIPAL COMPONENTS AND REGRESSION: A STATISTICAL EXPLANATION OF A NATURAL PHENOMENON , 2009 .

[17]  Xin Zhang,et al.  Fast Low-Rank Subspace Segmentation , 2014, IEEE Transactions on Knowledge and Data Engineering.

[18]  Yong Yu,et al.  Robust Subspace Segmentation by Low-Rank Representation , 2010, ICML.

[19]  David J. Kriegman,et al.  Clustering appearances of objects under varying illumination conditions , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[20]  Shuicheng Yan,et al.  Robust Subspace Segmentation with Block-Diagonal Prior , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Jorge Nocedal,et al.  A Stochastic Quasi-Newton Method for Large-Scale Optimization , 2014, SIAM J. Optim..

[22]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[24]  Tong Zhang,et al.  Solving large scale linear prediction problems using stochastic gradient descent algorithms , 2004, ICML.

[25]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[26]  Zhang Yi,et al.  Robust Subspace Clustering via Thresholding Ridge Regression , 2015, AAAI.

[27]  Ivica Kopriva,et al.  Multi-view low-rank sparse subspace clustering , 2017, Pattern Recognit..

[28]  Wanjun Chen,et al.  A Laplacian structured representation model in subspace clustering for enhanced motion segmentation , 2016, Neurocomputing.

[29]  Chris H. Q. Ding,et al.  K-Subspace Clustering , 2009, ECML/PKDD.

[30]  Lei Feng,et al.  Human Motion Segmentation via Robust Kernel Sparse Subspace Clustering , 2018, IEEE Transactions on Image Processing.

[31]  Dean P. Foster,et al.  Faster Ridge Regression via the Subsampled Randomized Hadamard Transform , 2013, NIPS.

[32]  Tommy W. S. Chow,et al.  Locality Constrained-ℓp Sparse Subspace Clustering for Image Clustering , 2016, Neurocomputing.

[33]  Guangcan Liu,et al.  Implicit Block Diagonal Low-Rank Representation , 2018, IEEE Transactions on Image Processing.

[34]  Ping Li,et al.  Online Low-Rank Subspace Clustering by Basis Dictionary Pursuit , 2015, ICML.

[35]  Zhang Yi,et al.  A Unified Framework for Representation-Based Subspace Clustering of Out-of-Sample and Large-Scale Data , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[36]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Piotr Indyk,et al.  Similarity Search in High Dimensions via Hashing , 1999, VLDB.

[38]  Dong Xu,et al.  FaLRR: A fast low rank representation solver , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).