Subspace Clustering via Good Neighbors

Finding the informative subspaces of high-dimensional datasets is at the core of numerous applications in computer vision, where spectral-based subspace clustering is arguably the most widely studied method due to its strong empirical performance. Such algorithms first compute an affinity matrix to construct a self-representation for each sample using other samples as a dictionary. Sparsity and connectivity of the self-representation play important roles in effective subspace clustering. However, simultaneous optimization of both factors is difficult due to their conflicting nature, and most existing methods are designed to address only one factor. In this paper, we propose a post-processing technique to optimize both sparsity and connectivity by finding good neighbors. Good neighbors induce key connections among samples within a subspace and not only have large affinity coefficients but are also strongly connected to each other. We reassign the coefficients of the good neighbors and eliminate other entries to generate a new coefficient matrix. We show that the few good neighbors can effectively recover the subspace, and the proposed post-processing step of finding good neighbors is complementary to most existing subspace clustering algorithms. Experiments on five benchmark datasets show that the proposed algorithm performs favorably against the state-of-the-art methods with negligible additional computation cost.

[1]  Zhao Kang,et al.  Subspace Clustering via Variance Regularized Ridge Regression , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Feiping Nie,et al.  Discriminatively Embedded K-Means for Multi-view Clustering , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Aswin C. Sankaranarayanan,et al.  Greedy feature selection for subspace clustering , 2013, J. Mach. Learn. Res..

[4]  Zhuwen Li,et al.  Simultaneous Clustering and Model Selection for Tensor Affinities , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Dong Xu,et al.  Learning Rotation-Invariant and Fisher Discriminative Convolutional Neural Networks for Object Detection , 2019, IEEE Transactions on Image Processing.

[6]  René Vidal,et al.  Low rank subspace clustering (LRSC) , 2014, Pattern Recognit. Lett..

[7]  Hans-Peter Kriegel,et al.  Subspace clustering , 2012, WIREs Data Mining Knowl. Discov..

[8]  Junbin Gao,et al.  Subspace Clustering for Sequential Data , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Shuicheng Yan,et al.  Nonconvex Nonsmooth Low Rank Minimization via Iteratively Reweighted Nuclear Norm , 2015, IEEE Transactions on Image Processing.

[10]  Qingshan Liu,et al.  Decentralized Robust Subspace Clustering , 2016, AAAI.

[11]  Junbin Gao,et al.  Laplacian Regularized Low-Rank Representation and Its Applications , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Yong Tang,et al.  Efficient k-Support Matrix Pursuit , 2014, ECCV.

[13]  Hongbin Zha,et al.  A Unified Convex Surrogate for the Schatten-p Norm , 2016, AAAI.

[14]  Feiping Nie,et al.  Clustering and projected clustering with adaptive neighbors , 2014, KDD.

[15]  Minsik Lee,et al.  Membership representation for detecting block-diagonal structure in low-rank or sparse subspace clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  George Atia,et al.  Innovation Pursuit: A New Approach to the Subspace Clustering Problem , 2017, ICML.

[17]  Huan Xu,et al.  Provable Subspace Clustering: When LRR Meets SSC , 2013, IEEE Transactions on Information Theory.

[18]  Kai Wang,et al.  Sub-GAN: An Unsupervised Generative Model via Subspaces , 2018, ECCV.

[19]  Shuicheng Yan,et al.  Correlation Adaptive Subspace Segmentation by Trace Lasso , 2013, 2013 IEEE International Conference on Computer Vision.

[20]  Junbin Gao,et al.  Dual Graph Regularized Latent Low-Rank Representation for Subspace Clustering , 2015, IEEE Transactions on Image Processing.

[21]  Tao Mei,et al.  Subspace Clustering by Block Diagonal Representation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[23]  George Atia,et al.  A Direction Search and Spectral Clustering Based Approach to Subspace Clustering , 2017, ArXiv.

[24]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Kai Wang,et al.  Automatic Model Selection in Subspace Clustering via Triplet Relationships , 2018, AAAI.

[26]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Daniel P. Robinson,et al.  Scalable Exemplar-based Subspace Clustering on Class-Imbalanced Data , 2018, European Conference on Computer Vision.

[28]  Junbin Gao,et al.  Kernel Sparse Subspace Clustering on Symmetric Positive Definite Manifolds , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Junwei Han,et al.  CNNs-Based RGB-D Saliency Detection via Cross-View Transfer and Multiview Fusion. , 2018, IEEE transactions on cybernetics.

[30]  Junwei Han,et al.  A Unified Metric Learning-Based Framework for Co-Saliency Detection , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Richard I. Hartley,et al.  Graph connectivity in sparse subspace clustering , 2011, CVPR 2011.

[32]  David P. Hofmeyr Clustering by Minimum Cut Hyperplanes , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Yin Wang,et al.  Subspace Clustering with Priors via Sparse Quadratically Constrained Quadratic Programming , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Sean Hughes,et al.  Clustering by Fast Search and Find of Density Peaks , 2016 .

[35]  Robert E. Tarjan,et al.  Depth-First Search and Linear Graph Algorithms , 1972, SIAM J. Comput..

[36]  Junbin Gao,et al.  Spatial subspace clustering for drill hole spectral data , 2014 .

[37]  Aarti Singh,et al.  Graph Connectivity in Noisy Sparse Subspace Clustering , 2015, AISTATS.

[38]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[39]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[40]  Shuicheng Yan,et al.  Efficient Subspace Segmentation via Quadratic Programming , 2011, AAAI.

[41]  Junbin Gao,et al.  Product Grassmann Manifold Representation and Its LRR Models , 2016, AAAI.

[42]  Constantine Caramanis,et al.  Greedy Subspace Clustering , 2014, NIPS.

[43]  René Vidal,et al.  A Structured Sparse Plus Structured Low-Rank Framework for Subspace Clustering and Completion , 2016, IEEE Transactions on Signal Processing.

[44]  Robert D. Nowak,et al.  The Information-Theoretic Requirements of Subspace Clustering with Missing Data , 2016, ICML.

[45]  Aarti Singh,et al.  Differentially private subspace clustering , 2015, NIPS.

[46]  Daniel P. Robinson,et al.  Provable Self-Representation Based Outlier Detection in a Union of Subspaces , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  René Vidal,et al.  Structured Sparse Subspace Clustering: A unified optimization framework , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Emmanuel J. Candès,et al.  A Geometric Analysis of Subspace Clustering with Outliers , 2011, ArXiv.

[49]  Zhang Yi,et al.  Robust Subspace Clustering via Thresholding Ridge Regression , 2015, AAAI.

[50]  Ehsan Elhamifar,et al.  Sparse subspace clustering , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Jörg H. Kappes,et al.  Fusion moves for correlation clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[52]  Deyu Meng,et al.  Leveraging Prior-Knowledge for Weakly Supervised Object Detection Under a Collaborative Self-Paced Curriculum Learning Framework , 2018, International Journal of Computer Vision.

[53]  Jianjiang Feng,et al.  Smooth Representation Clustering , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Shuicheng Yan,et al.  Robust Subspace Segmentation with Block-Diagonal Prior , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[56]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Ming-Hsuan Yang,et al.  Dynamic Match Kernel With Deep Convolutional Features for Image Retrieval , 2018, IEEE Transactions on Image Processing.

[58]  Songhwai Oh,et al.  Robust Elastic-Net Subspace Representation , 2016, IEEE Transactions on Image Processing.

[59]  Daniel P. Robinson,et al.  Oracle Based Active Set Algorithm for Scalable Elastic Net Subspace Clustering , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Qingshan Liu,et al.  A Deterministic Analysis for LRR , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Shuicheng Yan,et al.  Robust and Efficient Subspace Segmentation via Least Squares Regression , 2012, ECCV.

[62]  Nebojsa Jojic,et al.  -Sparse Subspace Clustering , 2016 .

[63]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[64]  Daniel P. Robinson,et al.  Scalable Sparse Subspace Clustering by Orthogonal Matching Pursuit , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[65]  Hongdong Li,et al.  Shape Interaction Matrix Revisited and Robustified: Efficient Subspace Clustering with Corrupted and Incomplete Data , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[66]  Dong Xu,et al.  FaLRR: A fast low rank representation solver , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[67]  Tat-Jun Chin,et al.  Clustering with Hypergraphs: The Case for Large Hyperedges , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[68]  Ke Chen,et al.  Reweighted sparse subspace clustering , 2015, Comput. Vis. Image Underst..

[69]  Zhang Yi,et al.  Scalable Sparse Subspace Clustering , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[70]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[71]  Zhang Yi,et al.  Constructing the L2-Graph for Robust Subspace Learning and Subspace Clustering , 2012, IEEE Transactions on Cybernetics.

[72]  Matthieu Cord,et al.  Closed-Form Training of Mahalanobis Distance for Supervised Clustering , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[73]  Huchuan Lu,et al.  Subspace clustering by Mixture of Gaussian Regression , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[74]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[75]  Dong Xu,et al.  Proximal Riemannian Pursuit for Large-Scale Trace-Norm Minimization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[76]  Wei Liu,et al.  Tensor Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Tensors via Convex Optimization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[77]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.