PMSSC: Parallelizable Multi-Subset based Self-Expressive Model for Subspace Clustering

Subspace clustering methods embrace a selfexpressive model that represents each data point as a linear combination of other data points in the dataset are powerful unsupervised learning techniques. However, when dealing with large-scale datasets, the representation of each data point by referring to all data points as a dictionary suffers from high computational complexity. To alleviate this issue, we introduce a parallelizable multi-subset based self-expressive model (PMS) which represents each data point by combing multiple subsets, with each consisting of only a small percentage of samples. The adoption of PMS in subspace clustering (PMSSC) leads to computational advantages because each optimization problem decomposed into each subset is small, and can be solved efficiently in parallel. Besides, PMSSC is able to combine multiple self-expressive coefficient vectors obtained from subsets, which contributes to the improvement of self-expressiveness. Extensive experiments on synthetic data and real-world datasets show the efficiency and effectiveness of our approach against competitive methods.

[1]  Shuicheng Yan,et al.  Robust and Efficient Subspace Segmentation via Least Squares Regression , 2012, ECCV.

[2]  Tong Zhang,et al.  Deep Subspace Clustering Networks , 2017, NIPS.

[3]  Yong Yu,et al.  Robust Subspace Segmentation by Low-Rank Representation , 2010, ICML.

[4]  Richard I. Hartley,et al.  Graph connectivity in sparse subspace clustering , 2011, CVPR 2011.

[5]  Chak-Kuen Wong,et al.  An Efficient Method for Weighted Sampling Without Replacement , 1980, SIAM J. Comput..

[6]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[7]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[8]  Tao Mei,et al.  Subspace Clustering by Block Diagonal Representation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Aswin C. Sankaranarayanan,et al.  Greedy feature selection for subspace clustering , 2013, J. Mach. Learn. Res..

[10]  Josef Kittler,et al.  Sparse subspace clustering via nonconvex approximation , 2018, Pattern Analysis and Applications.

[11]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[12]  Marc Pollefeys,et al.  A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate , 2006, ECCV.

[13]  Dejiao Zhang,et al.  Subspace Clustering using Ensembles of $K$-Subspaces , 2017, Information and Inference: A Journal of the IMA.

[14]  René Vidal,et al.  Adaptive Online k-Subspaces with Cooperative Re-Initialization , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[15]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[17]  Daniel P. Robinson,et al.  Oracle Based Active Set Algorithm for Scalable Elastic Net Subspace Clustering , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Hongdong Li,et al.  Efficient dense subspace clustering , 2014, IEEE Winter Conference on Applications of Computer Vision.

[19]  Farhad Kamangar,et al.  Multi-Level Representation Learning for Deep Subspace Clustering , 2020, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[20]  Junbin Gao,et al.  Subspace Clustering for Sequential Data , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[22]  Katsuya Hotta,et al.  Affine subspace clustering with nearest subspace neighbor , 2021, Other Conferences.

[23]  Chao ZHANG.  Energy Minimization over m-Branched Enumeration for Generalized Linear Subspace Clustering , 2019, IEICE Trans. Inf. Syst..

[24]  Allen Y. Yang,et al.  Unsupervised segmentation of natural images via lossy data compression , 2008, Comput. Vis. Image Underst..

[25]  Daniel P. Robinson,et al.  A divide-and-conquer framework for large-scale subspace clustering , 2016, 2016 50th Asilomar Conference on Signals, Systems and Computers.

[26]  U. Feige,et al.  Spectral Graph Theory , 2015 .

[27]  Stéphane Mallat,et al.  Invariant Scattering Convolution Networks , 2012, IEEE transactions on pattern analysis and machine intelligence.

[28]  Hans-Peter Kriegel,et al.  Subspace clustering , 2012, WIREs Data Mining Knowl. Discov..

[29]  Hongdong Li,et al.  Adaptive Low-Rank Kernel Subspace Clustering , 2017 .

[30]  Zhang Yi,et al.  Scalable Sparse Subspace Clustering , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Shin Matsushima,et al.  Selective Sampling-based Scalable Sparse Subspace Clustering , 2019, NeurIPS.

[32]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Guangliang Chen,et al.  Spectral Curvature Clustering (SCC) , 2009, International Journal of Computer Vision.

[34]  Yuxiao Hu,et al.  Learning a Spatially Smooth Subspace for Face Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Chong You,et al.  Stochastic Sparse Subspace Clustering , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[37]  Daniel P. Robinson,et al.  Scalable Exemplar-based Subspace Clustering on Class-Imbalanced Data , 2018, European Conference on Computer Vision.

[38]  Michael B. Wakin,et al.  Analysis of Orthogonal Matching Pursuit Using the Restricted Isometry Property , 2009, IEEE Transactions on Information Theory.

[39]  René Vidal,et al.  Multiframe Motion Segmentation with Missing Data Using PowerFactorization and GPCA , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[40]  Daniel P. Robinson,et al.  Scalable Sparse Subspace Clustering by Orthogonal Matching Pursuit , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Derek Greene,et al.  Practical solutions to the problem of diagonal dominance in kernel document clustering , 2006, ICML.

[42]  Gilad Lerman,et al.  Median K-Flats for hybrid linear modeling with many outliers , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[43]  Helmut Bölcskei,et al.  Subspace clustering via thresholding and spectral clustering , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[44]  P. Tseng.  Nearest q-Flat to m Points , 2000 .

[45]  Johannes Stallkamp,et al.  Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition , 2012, Neural Networks.

[46]  René Vidal,et al.  Low rank subspace clustering (LRSC) , 2014, Pattern Recognit. Lett..

[47]  D. Donoho.  For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[48]  Daniel P. Robinson,et al.  Self-Representation Based Unsupervised Exemplar Selection in a Union of Subspaces , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Chao Zhang,et al.  G2MF-WA: Geometric multi-model fitting with weakly annotated data , 2020, Computational Visual Media.