Robust Semi-Supervised Subspace Clustering via Non-Negative Low-Rank Representation

Low-rank representation (LRR) has been successfully applied in exploring the subspace structures of data. However, in previous LRR-based semi-supervised subspace clustering methods, the label information is not used to guide the affinity matrix construction so that the affinity matrix cannot deliver strong discriminant information. Moreover, these methods cannot guarantee an overall optimum since the affinity matrix construction and subspace clustering are often independent steps. In this paper, we propose a robust semi-supervised subspace clustering method based on non-negative LRR (NNLRR) to address these problems. By combining the LRR framework and the Gaussian fields and harmonic functions method in a single optimization problem, the supervision information is explicitly incorporated to guide the affinity matrix construction and the affinity matrix construction and subspace clustering are accomplished in one step to guarantee the overall optimum. The affinity matrix is obtained by seeking a non-negative low-rank matrix that represents each sample as a linear combination of others. We also explicitly impose the sparse constraint on the affinity matrix such that the affinity matrix obtained by NNLRR is non-negative low-rank and sparse. We introduce an efficient linearized alternating direction method with adaptive penalty to solve the corresponding optimization problem. Extensive experimental results demonstrate that NNLRR is effective in semi-supervised subspace clustering and robust to different types of noise than other state-of-the-art methods.

[1]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Ivor W. Tsang,et al.  Flexible Manifold Embedding: A Framework for Semi-Supervised and Unsupervised Dimension Reduction , 2010, IEEE Transactions on Image Processing.

[3]  Yong Luo,et al.  Multiview Vector-Valued Manifold Regularization for Multilabel Image Classification , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[4]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[5]  Junfeng Yang,et al.  A Fast Algorithm for Edge-Preserving Variational Multichannel Image Restoration , 2009, SIAM J. Imaging Sci..

[6]  Feiping Nie,et al.  Neighborhood MinMax Projections , 2007, IJCAI.

[7]  Xuelong Li,et al.  Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection , 2014, IEEE Transactions on Cybernetics.

[8]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[9]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[10]  John Wright,et al.  Principal Component Pursuit with reduced linear measurements , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[11]  Dacheng Tao,et al.  ReLISH: Reliable Label Inference via Smoothness Hypothesis , 2014, AAAI.

[12]  Feiping Nie,et al.  Optimal Mean Robust Principal Component Analysis , 2014, ICML.

[13]  Ting Wang,et al.  Kernel Sparse Representation-Based Classifier , 2012, IEEE Transactions on Signal Processing.

[14]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Ehsan Elhamifar,et al.  Sparse subspace clustering , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Jian Yang,et al.  Sparse Representation Classifier Steered Discriminative Projection With Applications to Face Recognition , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Xuelong Li,et al.  Data Uncertainty in Face Recognition , 2014, IEEE Transactions on Cybernetics.

[18]  Mikhail Belkin,et al.  Semi-Supervised Learning on Riemannian Manifolds , 2004, Machine Learning.

[19]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[20]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[21]  Jian Yang,et al.  Robust Subspace Segmentation Via Low-Rank Representation , 2014, IEEE Transactions on Cybernetics.

[22]  Dacheng Tao,et al.  Divide-and-Conquer Anchoring for Near-Separable Nonnegative Matrix Factorization and Completion in High Dimensions , 2013, 2013 IEEE 13th International Conference on Data Mining.

[23]  Shuicheng Yan,et al.  Latent Low-Rank Representation for subspace segmentation and feature extraction , 2011, 2011 International Conference on Computer Vision.

[24]  Zhixun Su,et al.  Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation , 2011, NIPS.

[25]  Nenghai Yu,et al.  Non-negative low rank and sparse graph for semi-supervised learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Yong Luo,et al.  Manifold Regularized Multitask Learning for Semi-Supervised Multilabel Image Classification , 2013, IEEE Transactions on Image Processing.

[27]  Marc Pollefeys,et al.  A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate , 2006, ECCV.

[28]  Feiping Nie,et al.  Clustering and projected clustering with adaptive neighbors , 2014, KDD.

[29]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[30]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[31]  Yao Zhao,et al.  Topographic NMF for Data Representation , 2014, IEEE Transactions on Cybernetics.

[32]  Feiping Nie,et al.  Orthogonal locality minimizing globality maximizing projections for feature extraction , 2009 .

[33]  Yong Xu,et al.  Locality and similarity preserving embedding for feature selection , 2014, Neurocomputing.

[34]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  TaoDacheng,et al.  Large-Margin Multi-ViewInformation Bottleneck , 2014 .

[36]  Dacheng Tao,et al.  Large-Margin Multi-ViewInformation Bottleneck , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Junbin Gao,et al.  Robust latent low rank representation for subspace clustering , 2014, Neurocomputing.

[38]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  René Vidal,et al.  Latent Space Sparse Subspace Clustering , 2013, 2013 IEEE International Conference on Computer Vision.

[40]  Josef Kittler,et al.  Incremental Linear Discriminant Analysis Using Sufficient Spanning Sets and Its Applications , 2010, International Journal of Computer Vision.

[41]  Dimitri P. Bertsekas Nonquadratic Penalty Functions — Convex Programming , 1982 .

[42]  S. Shankar Sastry,et al.  Generalized Principal Component Analysis , 2016, Interdisciplinary applied mathematics.

[43]  Jing-Yu Yang,et al.  Algebraic feature extraction for image recognition based on an optimal discriminant criterion , 1993, Pattern Recognit..