Discriminative and coherent subspace clustering

Abstract The ubiquitous large, complex and high dimensional datasets in computer vision and machine learning generate the problem of subspace clustering, which aims to partition the data into several low dimensional subspaces. Most state-of-the-art methods divide the problem into two stages: first learn the affinity from the data and then infer the cluster labels based on the affinity. The Structured Sparse Subspace Clustering (SSSC) model combines the affinity learning and the label inferring into one unified framework and empirically outperforms the two-stage methods. However, the SSSC method does not fully utilize the affinity and the labels to guide each other. In this work, we present a new regularity which combines the labels and the affinity to enforce the coherence of the affinity for data points from the same cluster and the discrimination of the labels for data points from different clusters. Based on this, we give a new unified optimization framework for subspace clustering. It enforces the coherence and discrimination of the affinity matrix as well as the labels, thus we call it Discriminative and Coherent Subspace Clustering (DCSC). Extended experiments on commonly used datasets demonstrate that our method performs better than some two stage state-of-the-art methods and the unified method SSSC in revealing the subspace structure of high-dimensional data.

[1]  Shuicheng Yan,et al.  Latent Low-Rank Representation for subspace segmentation and feature extraction , 2011, 2011 International Conference on Computer Vision.

[2]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Huan Xu,et al.  Provable Subspace Clustering: When LRR Meets SSC , 2013, IEEE Transactions on Information Theory.

[4]  Paul S. Bradley,et al.  k-Plane Clustering , 2000, J. Glob. Optim..

[5]  René Vidal,et al.  Multiframe Motion Segmentation with Missing Data Using PowerFactorization and GPCA , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[6]  Guangliang Chen,et al.  Spectral Curvature Clustering (SCC) , 2009, International Journal of Computer Vision.

[7]  P. Tseng Nearest q-Flat to m Points , 2000 .

[8]  René Vidal,et al.  Latent Space Sparse Subspace Clustering , 2013, 2013 IEEE International Conference on Computer Vision.

[9]  Huan Liu,et al.  Subspace clustering for high dimensional data: a review , 2004, SKDD.

[10]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Junbin Gao,et al.  Laplacian Regularized Low-Rank Representation and Its Applications , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Tieniu Tan,et al.  Robust Subspace Clustering via Half-Quadratic Minimization , 2013, 2013 IEEE International Conference on Computer Vision.

[13]  Svetha Venkatesh,et al.  Improved subspace clustering via exploitation of spatial constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Constantine Caramanis,et al.  Greedy Subspace Clustering , 2014, NIPS.

[15]  Michel Verleysen,et al.  Mixtures of robust probabilistic principal component analyzers , 2008, ESANN.

[16]  John Wright,et al.  Segmentation of Multivariate Mixed Data via Lossy Data Coding and Compression , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Jun Wang,et al.  LRSR: Low-Rank-Sparse representation for subspace clustering , 2016, Neurocomputing.

[18]  Nabil H. Mustafa,et al.  k-means projective clustering , 2004, PODS.

[19]  Allen Y. Yang,et al.  Estimation of Subspace Arrangements with Applications in Modeling and Segmenting Mixed Data , 2008, SIAM Rev..

[20]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Aswin C. Sankaranarayanan,et al.  Greedy feature selection for subspace clustering , 2013, J. Mach. Learn. Res..

[22]  Shuicheng Yan,et al.  Correlation Adaptive Subspace Segmentation by Trace Lasso , 2013, 2013 IEEE International Conference on Computer Vision.

[23]  Zhang Yi,et al.  Symmetric low-rank representation for subspace clustering , 2014, Neurocomputing.

[24]  Y. Weiss,et al.  Multibody factorization with uncertainty and missing data using the EM algorithm , 2004, CVPR 2004.

[25]  René Vidal,et al.  A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  René Vidal,et al.  Algebraic Clustering of Affine Subspaces , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Gilad Lerman,et al.  Hybrid Linear Modeling via Local Best-Fit Flats , 2010, International Journal of Computer Vision.

[28]  Shuicheng Yan,et al.  Robust and Efficient Subspace Segmentation via Least Squares Regression , 2012, ECCV.

[29]  Kun Huang,et al.  Multiscale Hybrid Linear Models for Lossy Image Representation , 2006, IEEE Transactions on Image Processing.

[30]  René Vidal,et al.  Kernel sparse subspace clustering , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[31]  Biao Hou,et al.  Discriminative Transformation Learning for Fuzzy Sparse Subspace Clustering , 2017, IEEE Transactions on Cybernetics.

[32]  René Vidal,et al.  Segmenting Motions of Different Types by Unsupervised Manifold Clustering , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  David J. Kriegman,et al.  Clustering appearances of objects under varying illumination conditions , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[34]  Shuicheng Yan,et al.  Robust Subspace Segmentation with Block-Diagonal Prior , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Yi Ma,et al.  Minimum effective dimension for mixtures of subspaces: a robust GPCA algorithm and its applications , 2004, CVPR 2004.

[36]  S. Sastry,et al.  An algebraic geometric approach to the identification of a class of linear hybrid systems , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[37]  Marc Pollefeys,et al.  A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate , 2006, ECCV.

[38]  Jianjiang Feng,et al.  Smooth Representation Clustering , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  René Vidal,et al.  Low rank subspace clustering (LRSC) , 2014, Pattern Recognit. Lett..

[40]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[41]  Ying Wu,et al.  Multibody Grouping by Inference of Multiple Subspaces from High-Dimensional Data Using Oriented-Frames , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  T. Boult,et al.  Factorization-based segmentation of motions , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[44]  Huchuan Lu,et al.  Subspace clustering by Mixture of Gaussian Regression , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[46]  Allen Y. Yang,et al.  Robust Statistical Estimation and Segmentation of Multiple Subspaces , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[47]  Allen Y. Yang,et al.  Unsupervised segmentation of natural images via lossy data compression , 2008, Comput. Vis. Image Underst..

[48]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Gilad Lerman,et al.  Median K-Flats for hybrid linear modeling with many outliers , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[50]  René Vidal,et al.  A closed form solution to robust subspace estimation and clustering , 2011, CVPR 2011.

[51]  Leonardo Torok,et al.  k-MS: A novel clustering algorithm based on morphological reconstruction , 2017, Pattern Recognit..

[52]  Helmut Bölcskei,et al.  Robust Subspace Clustering via Thresholding , 2013, IEEE Transactions on Information Theory.

[53]  René Vidal,et al.  Structured Sparse Subspace Clustering: A Joint Affinity Learning and Subspace Clustering Framework , 2016, IEEE Transactions on Image Processing.

[54]  Shuicheng Yan,et al.  Correntropy Induced L2 Graph for Robust Subspace Clustering , 2013, 2013 IEEE International Conference on Computer Vision.

[55]  René Vidal,et al.  Sparse subspace clustering , 2009, CVPR.

[56]  Ronen Basri,et al.  Lambertian Reflectance and Linear Subspaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  René Vidal,et al.  A Structured Sparse Plus Structured Low-Rank Framework for Subspace Clustering and Completion , 2016, IEEE Transactions on Signal Processing.