Simultaneous discriminative projection and dictionary learning for sparse representation based classification

Sparsity driven classification method has been popular recently due to its effectiveness in various classification tasks. It is based on the assumption that samples of the same class live in the same subspace, thus a test sample can be well represented by the training samples of the same class. Previous methods model the subspace for each class with either the training samples directly or dictionaries trained for each class separately. Although enabling strong reconstructive ability, these methods may not have desirable discriminative ability, especially when there are high correlations among the samples of different classes. In this paper, we propose to learn simultaneously a discriminative projection and a dictionary that are optimized for the sparse representation based classifier, to extract discriminative information from the raw data while respecting the sparse representation assumption. By formulating the task of projection and dictionary learning into an optimization framework, we can learn the discriminative projection and dictionary effectively. Extensive experiments are carried out on various datasets and the experimental results verify the efficacy of the proposed method.

[1]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[2]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[3]  J KriegmanDavid,et al.  Eigenfaces vs. Fisherfaces , 1997 .

[4]  Shuicheng Yan,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007 .

[5]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[6]  Ke Huang,et al.  Sparse Representation for Signal Classification , 2006, NIPS.

[7]  Rama Chellappa,et al.  Secure and Robust Iris Recognition Using Random Projections and Sparse Representations , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Thomas S. Huang,et al.  Heterogeneous multi-metric learning for multi-sensor fusion , 2011, 14th International Conference on Information Fusion.

[9]  Rajat Raina,et al.  Self-taught learning: transfer learning from unlabeled data , 2007, ICML '07.

[10]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[11]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[12]  I. Jolliffe Principal Component Analysis , 2002 .

[13]  David Zhang,et al.  Fisher Discrimination Dictionary Learning for sparse representation , 2011, 2011 International Conference on Computer Vision.

[14]  Roberto Paredes,et al.  Simultaneous learning of a discriminative projection and prototypes for Nearest-Neighbor classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  D. Donoho For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[17]  Guillermo Sapiro,et al.  Discriminative learned dictionaries for local image analysis , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Michael Elad,et al.  On the Role of Sparse and Redundant Representations in Image Processing , 2010, Proceedings of the IEEE.

[19]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Thomas S. Huang,et al.  Joint-Structured-Sparsity-Based Classification for Multiple-Measurement Transient Acoustic Signals , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[21]  Xudong Jiang,et al.  Linear Subspace Learning-Based Dimensionality Reduction , 2011, IEEE Signal Processing Magazine.

[22]  Thomas S. Huang,et al.  Close the loop: Joint blind image restoration and recognition with sparse representation prior , 2011, 2011 International Conference on Computer Vision.

[23]  Thomas S. Huang,et al.  Supervised translation-invariant sparse coding , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Enrique Vidal,et al.  Learning prototypes and distances (LPD). A prototype reduction technique based on nearest neighbor error minimization , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[25]  Larry S. Davis,et al.  Learning a discriminative dictionary for sparse coding via label consistent K-SVD , 2011, CVPR 2011.

[26]  Thomas S. Huang,et al.  Joint dynamic sparse representation for multi-view face recognition , 2012, Pattern Recognit..

[27]  Roberto Paredes,et al.  Dimensionality reduction by minimizing nearest-neighbor classification error , 2011, Pattern Recognit. Lett..

[28]  Thomas S. Huang,et al.  Multi-metric learning for multi-sensor fusion based classification , 2013, Inf. Fusion.

[29]  Xudong Jiang,et al.  Asymmetric Principal Component and Discriminant Analyses for Pattern Classification , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Jing Xiao,et al.  Non-negative matrix factorization as a feature selection tool for maximum margin classifiers , 2011, CVPR 2011.

[31]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[32]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, CVPR.

[34]  Guillermo Sapiro,et al.  Supervised Dictionary Learning , 2008, NIPS.

[35]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  David M. Bradley,et al.  Differentiable Sparse Coding , 2008, NIPS.

[37]  Xudong Jiang,et al.  Eigenfeature Regularization and Extraction in Face Recognition , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.