Scaled Simplex Representation for Subspace Clustering

The self-expressive property of data points, that is, each data point can be linearly represented by the other data points in the same subspace, has proven effective in leading subspace clustering (SC) methods. Most self-expressive methods usually construct a feasible affinity matrix from a coefficient matrix, obtained by solving an optimization problem. However, the negative entries in the coefficient matrix are forced to be positive when constructing the affinity matrix via exponentiation, absolute symmetrization, or squaring operations. This consequently damages the inherent correlations among the data. Besides, the affine constraint used in these methods is not flexible enough for practical applications. To overcome these problems, in this article, we introduce a scaled simplex representation (SSR) for the SC problem. Specifically, the non-negative constraint is used to make the coefficient matrix physically meaningful, and the coefficient vector is constrained to be summed up to a scalar s<1 to make it more discriminative. The proposed SSR-based SC (SSRSC) model is reformulated as a linear equality-constrained problem, which is solved efficiently under the alternating direction method of multipliers framework. Experiments on benchmark datasets demonstrate that the proposed SSRSC algorithm is very efficient and outperforms the state-of-the-art SC methods on accuracy. The code can be found at https://github.com/csjunxu/SSRSC.

[1]  David Zhang,et al.  Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[2]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[3]  Konstantinos Blekas,et al.  A novel framework for motion segmentation and tracking by clustering incomplete trajectories , 2012, Comput. Vis. Image Underst..

[4]  Shuicheng Yan,et al.  Robust and Efficient Subspace Segmentation via Least Squares Regression , 2012, ECCV.

[5]  Biao Hou,et al.  Discriminative Transformation Learning for Fuzzy Sparse Subspace Clustering , 2017, IEEE Transactions on Cybernetics.

[6]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[7]  Tong Zhang,et al.  Deep Subspace Clustering Networks , 2017, NIPS.

[8]  Takeo Kanade,et al.  A Multibody Factorization Method for Independently Moving Objects , 1998, International Journal of Computer Vision.

[9]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[10]  Daniel P. Robinson,et al.  Oracle Based Active Set Algorithm for Scalable Elastic Net Subspace Clustering , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Zenglin Xu,et al.  Clustering with Similarity Preserving , 2019, Neurocomputing.

[12]  René Vidal,et al.  A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  David Zhang,et al.  Patch Group Based Nonlocal Self-Similarity Prior Learning for Image Denoising , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[14]  Gilad Lerman,et al.  Hybrid Linear Modeling via Local Best-Fit Flats , 2010, International Journal of Computer Vision.

[15]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[16]  Ehsan Elhamifar,et al.  Sparse subspace clustering , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Ling Li,et al.  Affinity learning via a diffusion process for subspace clustering , 2018, Pattern Recognit..

[18]  René Vidal,et al.  Motion segmentation via robust subspace separation in the presence of outlying, incomplete, or corrupted trajectories , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Xuelong Li,et al.  Robust Semi-Supervised Subspace Clustering via Non-Negative Low-Rank Representation , 2016, IEEE Transactions on Cybernetics.

[20]  Zhao Kang,et al.  RES-PCA: A Scalable Approach to Recovering Low-Rank Matrices , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Huchuan Lu,et al.  Subspace clustering by Mixture of Gaussian Regression , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Laurent Condat.  Fast projection onto the simplex and the l1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pmb {l}_\mathbf {1}$$\end{ , 2015, Mathematical Programming.

[24]  Ivica Kopriva,et al.  𝓁0-Motivated Low-Rank Sparse Subspace Clustering , 2018, ArXiv.

[25]  Jianjiang Feng,et al.  Smooth Representation Clustering , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Kurt S. Riedel,et al.  A Sherman-Morrison-Woodbury Identity for Rank Augmenting Matrices with Application to Centering , 1992, SIAM J. Matrix Anal. Appl..

[27]  Zhang Yi,et al.  Robust Subspace Clustering via Thresholding Ridge Regression , 2015, AAAI.

[28]  Zhang Yi,et al.  Constructing the L2-Graph for Robust Subspace Learning and Subspace Clustering , 2012, IEEE Transactions on Cybernetics.

[29]  Junbin Gao,et al.  Ordered Subspace Clustering With Block-Diagonal Priors , 2016, IEEE Transactions on Cybernetics.

[30]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Marc Pollefeys,et al.  A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate , 2006, ECCV.

[32]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[33]  Nenghai Yu,et al.  Non-negative low rank and sparse graph for semi-supervised learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  David Zhang,et al.  A Trilateral Weighted Sparse Coding Scheme for Real-World Image Denoising , 2018, ECCV.

[35]  Hong Liu,et al.  Incomplete Multiview Spectral Clustering With Adaptive Graph Learning , 2020, IEEE Transactions on Cybernetics.

[36]  René Vidal,et al.  A closed form solution to robust subspace estimation and clustering , 2011, CVPR 2011.

[37]  Hongdong Li,et al.  Shape Interaction Matrix Revisited and Robustified: Efficient Subspace Clustering with Corrupted and Incomplete Data , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[38]  Zhang Yi,et al.  Scalable Sparse Subspace Clustering , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Zhao Kang,et al.  Subspace Clustering Using Log-determinant Rank Approximation , 2015, KDD.

[40]  C. Michelot.  A finite algorithm for finding the projection of a point onto the canonical simplex of ∝n , 1986 .

[41]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  S. Shankar Sastry,et al.  Dissimilarity-Based Sparse Subset Selection , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Feiping Nie,et al.  A New Simplex Sparse Learning Model to Measure Data Similarity for Clustering , 2015, IJCAI.

[44]  Yong Yu,et al.  Robust Subspace Segmentation by Low-Rank Representation , 2010, ICML.

[45]  Daniel P. Robinson,et al.  Provable Self-Representation Based Outlier Detection in a Union of Subspaces , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[47]  René Vidal,et al.  Structured Sparse Subspace Clustering: A Joint Affinity Learning and Subspace Clustering Framework , 2016, IEEE Transactions on Image Processing.

[48]  David Zhang,et al.  A Hybrid l1-l0 Layer Decomposition Model for Tone Mapping , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[49]  Stéphane Mallat,et al.  Invariant Scattering Convolution Networks , 2012, IEEE transactions on pattern analysis and machine intelligence.

[50]  René Vidal,et al.  Structured Sparse Subspace Clustering: A unified optimization framework , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Hans-Peter Kriegel,et al.  Subspace clustering , 2012, WIREs Data Mining Knowl. Discov..

[52]  Jian Yang,et al.  Robust Subspace Segmentation Via Low-Rank Representation , 2014, IEEE Transactions on Cybernetics.

[53]  Zhao Kang,et al.  Subspace Clustering via Variance Regularized Ridge Regression , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[54]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[55]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[56]  Hongchuan Yu,et al.  Constrained Low-Rank Representation for Robust Subspace Clustering , 2017, IEEE Transactions on Cybernetics.

[57]  Ivica Kopriva,et al.  $\ell_0$ -Motivated Low-Rank Sparse Subspace Clustering , 2018, IEEE Transactions on Cybernetics.

[58]  Gregory Cohen,et al.  EMNIST: Extending MNIST to handwritten letters , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[59]  Kai Wang,et al.  Automatic Model Selection in Subspace Clustering via Triplet Relationships , 2018, AAAI.

[60]  David Zhang,et al.  External Prior Guided Internal Prior Learning for Real-World Noisy Image Denoising , 2017, IEEE Transactions on Image Processing.

[61]  Daniel P. Robinson,et al.  Scalable Sparse Subspace Clustering by Orthogonal Matching Pursuit , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[62]  S. Shankar Sastry,et al.  Generalized Principal Component Analysis , 2016, Interdisciplinary applied mathematics.

[63]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[64]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[65]  Chong Peng,et al.  Discriminative Regression Machine: A Classifier for High-Dimensional Data or Imbalanced Data , 2019, ArXiv.

[66]  Daniel P. Robinson,et al.  Scalable Exemplar-based Subspace Clustering on Class-Imbalanced Data , 2018, ECCV 2018.

[67]  Zhao Kang,et al.  Kernel-driven similarity learning , 2017, Neurocomputing.

[68]  René Vidal,et al.  Algebraic Clustering of Affine Subspaces , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[69]  Yoram Singer,et al.  Efficient projections onto the l1-ball for learning in high dimensions , 2008, ICML '08.

[70]  Zenglin Xu,et al.  Low-rank kernel learning for graph-based clustering , 2019, Knowl. Based Syst..

[71]  Zenglin Xu,et al.  Robust Graph Learning From Noisy Data , 2018, IEEE Transactions on Cybernetics.

[72]  Jinhui Tang,et al.  Constructing a Nonnegative Low-Rank and Sparse Graph With Data-Adaptive Features , 2014, IEEE Transactions on Image Processing.

[73]  Shuicheng Yan,et al.  Robust Subspace Segmentation with Block-Diagonal Prior , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[74]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[75]  Gregory Cohen,et al.  EMNIST: an extension of MNIST to handwritten letters , 2017, CVPR 2017.

[76]  T. Lumley,et al.  PRINCIPAL COMPONENT ANALYSIS AND FACTOR ANALYSIS , 2004, Statistical Methods for Biomedical Research.

[77]  Zhao Kang,et al.  Integrating feature and graph learning with low-rank representation , 2017, Neurocomputing.

[78]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[79]  David Zhang,et al.  Sparse, Collaborative, or Nonnegative Representation: Which Helps Pattern Classification? , 2018, Pattern Recognition.

[80]  Minsik Lee,et al.  Nonparametric Estimation of Probabilistic Membership for Subspace Clustering , 2020, IEEE Transactions on Cybernetics.

[81]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[82]  Jian Yang,et al.  LRR for Subspace Segmentation via Tractable Schatten- $p$ Norm Minimization and Factorization , 2019, IEEE Transactions on Cybernetics.

[83]  James M. Ferryman,et al.  Biologically-inspired robust motion segmentation using mutual information , 2014, Comput. Vis. Image Underst..

[84]  Yair Weiss,et al.  Multibody factorization with uncertainty and missing data using the EM algorithm , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[85]  Qiang Cheng,et al.  Discriminative Ridge Machine: A Classifier for High-Dimensional Data or Imbalanced Data. , 2020, IEEE transactions on neural networks and learning systems.

[86]  Ke Chen,et al.  Reweighted sparse subspace clustering , 2015, Comput. Vis. Image Underst..

[87]  David J. Kriegman,et al.  Clustering appearances of objects under varying illumination conditions , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[88]  Junbin Gao,et al.  Subspace Clustering via Learning an Adaptive Low-Rank Graph , 2018, IEEE Transactions on Image Processing.

[89]  R. Courant.  Variational methods for the solution of problems of equilibrium and vibrations , 1943 .

[90]  Junbin Gao,et al.  Dual Graph Regularized Latent Low-Rank Representation for Subspace Clustering , 2015, IEEE Transactions on Image Processing.

[91]  René Vidal,et al.  Segmenting Motions of Different Types by Unsupervised Manifold Clustering , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[92]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..