Robust Alternating Low-Rank Representation by joint Lp- and L2, p-norm minimization

We propose a robust Alternating Low-Rank Representation (ALRR) model formed by an alternating forward-backward representation process. For forward representation, ALRR first recovers the low-rank PCs and random corruptions by an adaptive local Robust PCA (RPCA). Then, ALRR performs a joint Lp-norm and L2,p-norm minimization (0<p <1) based sparse LRR by taking the low-rank PCs as inputs and dictionary instead of using the original noisy data to learn the coding coefficients for subspace recovery, where the Lp-norm on the coefficients can ensure joint sparsity for subspace representation, while the L2,p-norm on the reconstruction error can handle outlier pursuit. After that, ALRR returns the coefficients as adaptive weights to local RPCA for updating PCs and dictionary in the backward representation process. Thus, ALRR is regarded as an integration of local RPCA with adaptive weights plus sparse LRR with a self-expressive low-rank dictionary. To enable ALRR to handle outside data efficiently, a projective ALRR that can extract features from data directly by embedding is also derived. To solve the L2,p-norm based minimization problem, a new iterative scheme based on the Iterative Shrinkage/Thresholding (IST) approach is presented. The relationship analysis with other related criteria show that our methods are more general. Visual and numerical results demonstrate the effectiveness of our algorithms for representation.

[1]  Jing Liu,et al.  Robust Structured Subspace Learning for Data Representation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Y. She,et al.  Thresholding-based iterative selection procedures for model selection and shrinkage , 2008, 0812.5061.

[3]  Shuicheng Yan,et al.  Active Subspace: Toward Scalable Low-Rank Learning , 2012, Neural Computation.

[4]  Nenghai Yu,et al.  Non-negative low rank and sparse graph for semi-supervised learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Yulong Wang,et al.  Graph-Regularized Low-Rank Representation for Destriping of Hyperspectral Images , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Arvind Ganesh,et al.  Fast Convex Optimization Algorithms for Exact Recovery of a Corrupted Low-Rank Matrix , 2009 .

[7]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[8]  Shuicheng Yan,et al.  Generalized Singular Value Thresholding , 2014, AAAI.

[9]  Shuicheng Yan,et al.  Latent Low-Rank Representation for subspace segmentation and feature extraction , 2011, 2011 International Conference on Computer Vision.

[10]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Martin Vetterli,et al.  Compressive Sampling [From the Guest Editors] , 2008, IEEE Signal Processing Magazine.

[12]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Xuelong Li,et al.  Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection , 2014, IEEE Transactions on Cybernetics.

[14]  Xiaodong Li,et al.  Stable Principal Component Pursuit , 2010, 2010 IEEE International Symposium on Information Theory.

[15]  Feiping Nie,et al.  Joint Schatten $$p$$p-norm and $$\ell _p$$ℓp-norm robust matrix completion for missing value recovery , 2013, Knowledge and Information Systems.

[16]  Songcan Chen,et al.  A unified algorithm for mixed $$l_{2,p}$$l2,p-minimizations and its application in feature selection , 2014, Comput. Optim. Appl..

[17]  T. Hastie,et al.  SparseNet: Coordinate Descent With Nonconvex Penalties , 2011, Journal of the American Statistical Association.

[18]  Zhao Zhang,et al.  Joint lp- and l2,p-norm minimization for subspace clustering with outlier pursuit , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[19]  Feiping Nie,et al.  Robust Principal Component Analysis with Non-Greedy l1-Norm Maximization , 2011, IJCAI.

[20]  E.J. Candes.  Compressive Sampling , 2022 .

[21]  Constantine Caramanis,et al.  Robust PCA via Outlier Pursuit , 2010, IEEE Transactions on Information Theory.

[22]  Yong Yu,et al.  Robust Subspace Segmentation by Low-Rank Representation , 2010, ICML.

[23]  Shuicheng Yan,et al.  Similarity preserving low-rank representation for enhanced data representation and effective subspace learning , 2014, Neural Networks.

[24]  Loong Fah Cheong,et al.  Robust Low-Rank Subspace Segmentation with Semidefinite Guarantees , 2010, 2010 IEEE International Conference on Data Mining Workshops.

[25]  Victor Vianu,et al.  Invited articles section foreword , 2010, JACM.

[26]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  R. Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications. , 2013, IEEE transactions on pattern analysis and machine intelligence.

[28]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[29]  D. Donoho.  For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[30]  Yiyuan She,et al.  Thresholding-based Iterative Selection Procedures for Generalized Linear Models , 2009, 0911.5460.

[31]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Dacheng Tao,et al.  GoDec: Randomized Lowrank & Sparse Matrix Decomposition in Noisy Case , 2011, ICML.

[33]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[35]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[36]  Xiaoyang Tan,et al.  Pattern Recognition , 2016, Communications in Computer and Information Science.

[37]  René Vidal,et al.  A closed form solution to robust subspace estimation and clustering , 2011, CVPR 2011.

[38]  El-hadi Zahzah,et al.  Robust Principal Component Analysis Based on Low-Rank and Block-Sparse Matrix Decomposition , 2016 .

[39]  Changsheng Xu,et al.  Inductive Robust Principal Component Analysis , 2012, IEEE Transactions on Image Processing.

[40]  Shuicheng Yan,et al.  Pairwise Sparsity Preserving Embedding for Unsupervised Subspace Learning and Classification , 2013, IEEE Transactions on Image Processing.

[41]  Zhixun Su,et al.  Fixed-rank representation for unsupervised visual learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Qingshan Liu,et al.  A Deterministic Analysis for LRR , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  John Wright,et al.  Segmentation of Multivariate Mixed Data via Lossy Data Coding and Compression , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Li Zhang,et al.  Joint Low-Rank and Sparse Principal Feature Coding for Enhanced Robust Representation and Visual Classification , 2016, IEEE Transactions on Image Processing.

[45]  Shuicheng Yan,et al.  Bilinear low-rank coding framework and extension for robust image recovery and feature representation , 2015, Knowl. Based Syst..

[46]  Li Zhang,et al.  Semi-Supervised Image Classification by Nonnegative Sparse Neighborhood Propagation , 2015, ICMR.

[47]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[48]  Feiping Nie,et al.  Robust Matrix Completion via Joint Schatten p-Norm and lp-Norm Minimization , 2012, 2012 IEEE 12th International Conference on Data Mining.

[49]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[50]  Tommy W. S. Chow,et al.  Binary- and Multi-class Group Sparse Canonical Correlation Analysis for Feature Extraction and Classification , 2013, IEEE Transactions on Knowledge and Data Engineering.

[51]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[52]  Nojun Kwak,et al.  Principal Component Analysis Based on L1-Norm Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.