Dictionary learning enhancement framework: Learning a non-linear mapping model to enhance discriminative dictionary learning methods

Abstract In this paper, a new framework is presented to enhance the reconstruction and discrimination capabilities of existing discriminative dictionary learning methods. In the proposed framework, a non-linear mapping model is introduced to learn a feature space in a way that any standard discriminative dictionary learning algorithms could achieve higher classification accuracies. The proposed feature mapping process targets to boost the standard dictionary learning methods by facilitating their optimization process. The mapping model uses a modified autoencoder network to provide a higher level of reconstruction and discrimination capabilities for the discriminative dictionary learning methods. The proposed dictionary learning enhancement (DLE) framework could be applied to any discriminative dictionary learning methods with the embedded discriminative term in their objective functions. Our experiments on several real-world image datasets demonstrate that the proposed framework could improve the classification accuracies of standard discriminative dictionary learning methods.

[1]  Yi Ma,et al.  Learning Category-Specific Dictionary and Shared Dictionary for Fine-Grained Image Categorization , 2014, IEEE Transactions on Image Processing.

[2]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[3]  Thomas S. Huang,et al.  Simultaneous discriminative projection and dictionary learning for sparse representation based classification , 2013, Pattern Recognit..

[4]  Jianping Fan,et al.  Image collection summarization via dictionary learning for sparse representation , 2013, Pattern Recognit..

[5]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[6]  Svetlana Lazebnik,et al.  Supervised Learning of Quantizer Codebooks by Information Loss Minimization , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[8]  Liang-Tien Chia,et al.  Kernel Sparse Representation for Image Classification and Face Recognition , 2010, ECCV.

[9]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Ting Wang,et al.  Kernel Sparse Representation-Based Classifier , 2012, IEEE Transactions on Signal Processing.

[11]  Hideo Hashimoto,et al.  Matching pursuits using slant patterns and its dictionary design [video coding applications] , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[12]  Pascal Vincent,et al.  Kernel Matching Pursuit , 2002, Machine Learning.

[13]  Qinghua Hu,et al.  Kernel sparse representation for time series classification , 2015, Inf. Sci..

[14]  Larry S. Davis,et al.  Label Consistent K-SVD: Learning a Discriminative Dictionary for Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Bruce A. Draper,et al.  Analyzing PCA-based Face Recognition Algorithms: Eigenvector Selection and Distance Measures , 2003 .

[16]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[17]  Gabriel Peyré,et al.  Sparse Modeling of Textures , 2009, Journal of Mathematical Imaging and Vision.

[18]  Rajat Raina,et al.  Classification with Hybrid Generative/Discriminative Models , 2003, NIPS.

[19]  Ivor W. Tsang,et al.  Improved Nyström low-rank approximation and error analysis , 2008, ICML '08.

[20]  Edoardo Amaldi,et al.  On the Approximability of Minimizing Nonzero Variables or Unsatisfied Relations in Linear Systems , 1998, Theor. Comput. Sci..

[21]  Shuyuan Yang,et al.  Multitask dictionary learning and sparse representation based single-image super-resolution reconstruction , 2011, Neurocomputing.

[22]  Matthias W. Seeger,et al.  Using the Nyström Method to Speed Up Kernel Machines , 2000, NIPS.

[23]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[24]  Rama Chellappa,et al.  Kernel dictionary learning , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[25]  Qingshan Liu,et al.  Learning Discriminative Dictionary for Group Sparse Representation , 2014, IEEE Transactions on Image Processing.

[26]  Stephen J. Wright,et al.  Sparse reconstruction by separable approximation , 2009, IEEE Trans. Signal Process..

[27]  Meng Yang,et al.  Joint kernel dictionary and classifier learning for sparse coding via locality preserving K-SVD , 2015, 2015 IEEE International Conference on Multimedia and Expo (ICME).

[28]  Stéphane Canu,et al.  Kernel Basis Pursuit , 2005, ECML.

[29]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[30]  Tom Minka,et al.  Principled Hybrids of Generative and Discriminative Models , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[31]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[32]  Kjersti Engan,et al.  Method of optimal directions for frame design , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[33]  Anastasios Tefas,et al.  Entropy Optimized Feature-Based Bag-of-Words Representation for Information Retrieval , 2016, IEEE Transactions on Knowledge and Data Engineering.

[34]  Michael Elad,et al.  Sparse and redundant representations and motion-estimation-free algorithm for video denoising , 2007, SPIE Optical Engineering + Applications.

[35]  Matthijs Douze,et al.  How should we evaluate supervised hashing? , 2016, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[36]  Donghui Wang,et al.  A classification-oriented dictionary learning model: Explicitly learning the particularity and commonality across categories , 2014, Pattern Recognit..

[37]  Asok Ray,et al.  Multimodal Task-Driven Dictionary Learning for Image Classification , 2015, IEEE Transactions on Image Processing.

[38]  S. Mallat A wavelet tour of signal processing , 1998 .

[39]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003 .

[40]  Baoxin Li,et al.  Discriminative K-SVD for dictionary learning in face recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[41]  Minh N. Do,et al.  Contourlets: a directional multiresolution image representation , 2002, Proceedings. International Conference on Image Processing.

[42]  Jiang Li,et al.  A new decision rule for sparse representation based classification for face recognition , 2013, Neurocomputing.

[43]  Trevor Hastie,et al.  Linear Methods for Regression , 2001 .

[44]  Rama Chellappa,et al.  Design of Non-Linear Discriminative Dictionaries for Image Classification , 2012, ACCV.

[45]  Michael Elad,et al.  On the Role of Sparse and Redundant Representations in Image Processing , 2010, Proceedings of the IEEE.

[46]  Ming-Hsuan Yang,et al.  Top-down visual saliency via joint CRF and dictionary learning , 2012, CVPR.

[47]  Michael Elad,et al.  Linearized Kernel Dictionary Learning , 2015, IEEE Journal of Selected Topics in Signal Processing.

[48]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[49]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[50]  Ajmal S. Mian,et al.  Discriminative Bayesian Dictionary Learning for Classification , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Kai Liu,et al.  Kernel-based sparse representation for gesture recognition , 2013, Pattern Recognit..

[52]  A. Martínez,et al.  The AR face databasae , 1998 .

[53]  Bernhard Schölkopf,et al.  Kernel Principal Component Analysis , 1997, ICANN.

[54]  Guillermo Sapiro,et al.  Discriminative learned dictionaries for local image analysis , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[56]  E. Candès,et al.  Recovering edges in ill-posed inverse problems: optimality of curvelet frames , 2002 .

[57]  Guillermo Sapiro,et al.  Supervised Dictionary Learning , 2008, NIPS.

[58]  Mohamed S. Kamel,et al.  Kernelized Supervised Dictionary Learning , 2012, IEEE Transactions on Signal Processing.

[59]  Hossein Mobahi,et al.  Toward a Practical Face Recognition System: Robust Alignment and Illumination by Sparse Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  Rama Chellappa,et al.  Information-Theoretic Dictionary Learning for Image Classification , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[62]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[63]  D. Donoho For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[64]  Christian Jutten,et al.  K-LDA: An algorithm for learning jointly overcomplete and discriminative dictionaries , 2014, 2014 22nd European Signal Processing Conference (EUSIPCO).

[65]  Stefan Winkler,et al.  Category Specific Dictionary Learning for Attribute Specific Feature Selection , 2016, IEEE Transactions on Image Processing.