Learning Fast Low-Rank Projection for Image Classification

Rooted in a basic hypothesis that a data matrix is strictly drawn from some independent subspaces, the low-rank representation (LRR) model and its variations have been successfully applied in various image classification tasks. However, this hypothesis is very strict to the LRR model as it cannot always be guaranteed in real images. Moreover, the hypothesis also prevents the sub-dictionaries of different subspaces from collaboratively representing an image. Fortunately, in supervised image classification, low-rank signal can be extracted from the independent label subspaces (ILS) instead of the independent image subspaces (IIS). Therefore, this paper proposes a projective low-rank representation (PLR) model by directly training a projective function to approximate the LRR derived from the labels. To the best of our knowledge, PLR is the first attempt to use the ILS hypothesis to relax the rigorous IIS hypothesis in the LRR models. We further prove a low-rank effect that the representations learned by PLR have high intraclass similarities and large interclass differences, which are beneficial to the classification tasks. The effectiveness of our proposed approach is validated by the experimental results on three databases.

[1]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[2]  Jian Yang,et al.  Learning discriminative low-rank representation for image classification , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[3]  Yun Fu,et al.  Learning low-rank and discriminative dictionary for image classification , 2014, Image Vis. Comput..

[4]  Marc'Aurelio Ranzato,et al.  Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition , 2010, ArXiv.

[5]  Jieping Ye,et al.  A shared-subspace learning framework for multi-label classification , 2010, TKDD.

[6]  Changsheng Xu,et al.  Low-Rank Sparse Coding for Image Classification , 2013, 2013 IEEE International Conference on Computer Vision.

[7]  Dacheng Tao,et al.  Algorithm-Dependent Generalization Bounds for Multi-Task Learning , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Zhi-Quan Luo,et al.  Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems , 2014, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[9]  C. Févotte,et al.  Automatic Relevance Determination in Nonnegative Matrix Factorization with the-Divergence , 2011 .

[10]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[12]  Dong Xu,et al.  Dimensionality-Dependent Generalization Bounds for k-Dimensional Coding Schemes , 2016, Neural Computation.

[13]  Jie Gui,et al.  Factor Analysis for Cross-Platform Tumor Classification Based on Gene Expression Profiles , 2010, J. Circuits Syst. Comput..

[14]  Yun Fu,et al.  Learning Balanced and Unbalanced Graphs via Low-Rank Coding , 2015, IEEE Transactions on Knowledge and Data Engineering.

[15]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[16]  LinLin Shen,et al.  Analysis-Synthesis Dictionary Learning for Universality-Particularity Representation Based Classification , 2016, AAAI.

[17]  Ping Guo,et al.  Locality-Constrained Low-Rank Coding for Image Classification , 2014, AAAI.

[18]  Dacheng Tao,et al.  Large-Cone Nonnegative Matrix Factorization , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Ying Tai,et al.  Nuclear Norm Based Matrix Regression with Applications to Face Recognition with Occlusion and Illumination Changes , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Vincent Y. F. Tan,et al.  Automatic Relevance Determination in Nonnegative Matrix Factorization with the /spl beta/-Divergence , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Ming Shao,et al.  Generalized Transfer Subspace Learning Through Low-Rank Constraint , 2014, International Journal of Computer Vision.

[22]  Shuicheng Yan,et al.  Multi-task low-rank affinity pursuit for image segmentation , 2011, 2011 International Conference on Computer Vision.

[23]  David Zhang,et al.  Sparse Representation Based Fisher Discrimination Dictionary Learning for Image Classification , 2014, International Journal of Computer Vision.

[24]  Tieniu Tan,et al.  Representative Vector Machines: A Unified Framework for Classical Classifiers , 2016, IEEE Transactions on Cybernetics.

[25]  Chunheng Wang,et al.  Sparse representation for face recognition based on discriminative low-rank dictionary learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Wei Jia,et al.  Locality preserving discriminant projections for face and palmprint recognition , 2010, Neurocomputing.

[27]  Xuelong Li,et al.  Discriminative Transfer Subspace Learning via Low-Rank and Sparse Representation , 2016, IEEE Transactions on Image Processing.

[28]  Fuchun Sun,et al.  Supervised Low-Rank Matrix Recovery for Traffic Sign Recognition in Image Sequences , 2013, IEEE Signal Processing Letters.

[29]  Yann LeCun,et al.  Learning Fast Approximations of Sparse Coding , 2010, ICML.

[30]  Yong Luo,et al.  Group Sparse Multiview Patch Alignment Framework With View Consistency for Image Classification , 2014, IEEE Transactions on Image Processing.

[31]  Qi Tian,et al.  Image classification by non-negative sparse coding, low-rank and sparse decomposition , 2011, CVPR 2011.

[32]  Jie Zhang,et al.  Structure-Constrained Low-Rank Representation , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[33]  Jian Yang,et al.  Sparseness Analysis in the Pretraining of Deep Neural Networks , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Xiaochun Cao,et al.  Low-Rank Tensor Constrained Multiview Subspace Clustering , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[35]  Xiaojun Wu,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Dacheng Tao,et al.  Classification with Noisy Labels by Importance Reweighting , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Shuicheng Yan,et al.  Robust Subspace Segmentation with Block-Diagonal Prior , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Gianfranco Doretto,et al.  A Supervised Low-Rank Method for Learning Invariant Subspaces , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[39]  Shimon Ullman,et al.  Uncovering shared structures in multiclass classification , 2007, ICML '07.

[40]  Ziqiang Shi,et al.  Audio Segment Classification Using Online Learning Based Tensor Representation Feature Discrimination , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[41]  Xudong Jiang,et al.  Classwise Sparse and Collaborative Patch Representation for Face Recognition , 2016, IEEE Trans. Image Process..

[42]  Tieniu Tan,et al.  Feature Selection Based on Structured Sparsity: A Comprehensive Study , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[43]  Xuelong Li,et al.  Robust Semi-Supervised Subspace Clustering via Non-Negative Low-Rank Representation , 2016, IEEE Transactions on Cybernetics.

[44]  Ming Shao,et al.  Latent Low-Rank Transfer Subspace Learning for Missing Modality Recognition , 2014, AAAI.

[45]  Larry S. Davis,et al.  Learning Structured Low-Rank Representations for Image Classification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Jian Yang,et al.  Sparse Deep Stacking Network for Image Classification , 2015, AAAI.

[47]  Yun Fu,et al.  Robust Subspace Discovery through Supervised Low-Rank Constraints , 2014, SDM.

[48]  Larry S. Davis,et al.  Label Consistent K-SVD: Learning a Discriminative Dictionary for Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Larry S. Davis,et al.  Multi-Task Learning with Low Rank Attribute Embedding for Person Re-Identification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[51]  Bernhard Schölkopf,et al.  Domain Adaptation with Conditional Transferable Components , 2016, ICML.

[52]  Shuicheng Yan,et al.  Latent Low-Rank Representation for subspace segmentation and feature extraction , 2011, 2011 International Conference on Computer Vision.

[53]  Zhixun Su,et al.  Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation , 2011, NIPS.

[54]  Chao Zhang,et al.  Integrated Low-Rank-Based Discriminative Feature Learning for Recognition , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[55]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[56]  D. Donoho For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[57]  Yun Fu,et al.  Low-Rank Common Subspace for Multi-view Learning , 2014, 2014 IEEE International Conference on Data Mining.

[58]  Lei Zhang,et al.  Sparse representation or collaborative representation: Which helps face recognition? , 2011, 2011 International Conference on Computer Vision.

[59]  Dong Xu,et al.  FaLRR: A fast low rank representation solver , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Jing Liu,et al.  Learning Robust Face Representation With Classwise Block-Diagonal Structure , 2014, IEEE Transactions on Information Forensics and Security.

[61]  Yun Fu,et al.  Learning Robust and Discriminative Subspace With Low-Rank Constraints , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[62]  Jieping Ye,et al.  Learning Incoherent Sparse and Low-Rank Patterns from Multiple Tasks , 2010, TKDD.

[63]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.