Non-negative Local Sparse Coding for Subspace Clustering

Subspace sparse coding (SSC) algorithms have proven to be beneficial to the clustering problems. They provide an alternative data representation in which the underlying structure of the clusters can be better captured. However, most of the research in this area is mainly focused on enhancing the sparse coding part of the problem. In contrast, we introduce a novel objective term in our proposed SSC framework which focuses on the separability of data points in the coding space. We also provide mathematical insights into how this local-separability term improves the clustering result of the SSC framework. Our proposed non-linear local SSC algorithm (NLSSC) also benefits from the efficient choice of its sparsity terms and constraints. The NLSSC algorithm is also formulated in the kernel-based framework (NLKSSC) which can represent the nonlinear structure of data. In addition, we address the possibility of having redundancies in sparse coding results and its negative effect on graph-based clustering problems. We introduce the link-restore post-processing step to improve the representation graph of non-negative SSC algorithms such as ours. Empirical evaluations on well-known clustering benchmarks show that our proposed NLSSC framework results in better clusterings compared to the state-of-the-art baselines and demonstrate the effectiveness of the link-restore post-processing in improving the clustering accuracy via correcting the broken links of the representation graph.

[1]  Patrik O. Hoyer,et al.  Modeling Receptive Fields with Non-Negative Sparse Coding , 2002, Neurocomputing.

[2]  René Vidal,et al.  Low rank subspace clustering (LRSC) , 2014, Pattern Recognit. Lett..

[3]  Gregory Shakhnarovich,et al.  Sparse Coding for Learning Interpretable Spatio-Temporal Primitives , 2010, NIPS.

[4]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[5]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[7]  René Vidal,et al.  Structured Sparse Subspace Clustering: A unified optimization framework , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Liang-Tien Chia,et al.  Laplacian Sparse Coding, Hypergraph Laplacian Sparse Coding, and Applications , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Dong Xu,et al.  Robust Kernel Low-Rank Representation , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Nenghai Yu,et al.  Non-negative low rank and sparse graph for semi-supervised learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  René Vidal,et al.  Kernel sparse subspace clustering , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[12]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[13]  Daniel P. Robinson,et al.  Scalable Sparse Subspace Clustering by Orthogonal Matching Pursuit , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[15]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[17]  Ana L. N. Fred,et al.  Robust data clustering , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[18]  James M. Rehg,et al.  Beyond the Euclidean distance: Creating effective visual codebooks using the Histogram Intersection Kernel , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[19]  Xianfeng Li,et al.  A novel cluster ensemble approach effected by subspace similarity , 2016, Intell. Data Anal..

[20]  Michael Elad,et al.  Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit , 2008 .

[21]  A. Martínez,et al.  The AR face databasae , 1998 .

[22]  Shuicheng Yan,et al.  Learning With $\ell ^{1}$-Graph for Image Analysis , 2010, IEEE Transactions on Image Processing.

[23]  Jiangping Wang,et al.  Data Clustering by Laplacian Regularized L1-Graph , 2014, AAAI.

[24]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[25]  Peter Kirrinnis,et al.  Fast algorithms for the Sylvester equation AX-XBT=C , 2001, Theor. Comput. Sci..

[26]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[27]  Feng Li,et al.  Kernelized Sparse Self-Representation for Clustering and Recommendation , 2016, SDM.

[28]  Xuelong Li,et al.  Graph Regularized Non-Negative Low-Rank Matrix Factorization for Image Clustering , 2017, IEEE Transactions on Cybernetics.

[29]  Roberto Tron RenVidal A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms , 2007 .