Active Dictionary Learning in Sparse Representation Based Classification

Sparse representation, which uses dictionary atoms to reconstruct input vectors, has been studied intensively in recent years. A proper dictionary is a key for the success of sparse representation. In this paper, an active dictionary learning (ADL) method is introduced, in which classification error and reconstruction error are considered as the active learning criteria in selection of the atoms for dictionary construction. The learned dictionaries are caculated in sparse representation based classification (SRC). The classification accuracy and reconstruction error are used to evaluate the proposed dictionary learning method. The performance of the proposed dictionary learning method is compared with other methods, including unsupervised dictionary learning and whole-training-data dictionary. The experimental results based on the UCI data sets and face data set demonstrate the efficiency of the proposed method.

[1]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Jing Wang,et al.  A Hierarchical Neural Network Architecture for Classification , 2012, ISNN.

[3]  Stéphane Mallat,et al.  Sparse geometric image representations with bandelets , 2005, IEEE Transactions on Image Processing.

[4]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[6]  Haibo He,et al.  DCPE co-training: Co-training based on diversity of class probability estimation , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[7]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[8]  Guang Yang,et al.  L 1 Graph Based on Sparse Coding for Feature Selection , 2013, ISNN.

[9]  Ian Davidson,et al.  Active Spectral Clustering , 2010, 2010 IEEE International Conference on Data Mining.

[10]  Kun Deng,et al.  Active Learning from Multiple Noisy Labelers with Varied Costs , 2010, 2010 IEEE International Conference on Data Mining.

[11]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, CVPR.

[12]  Baoxin Li,et al.  Discriminative K-SVD for dictionary learning in face recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Jin Xu,et al.  Dictionary Learning Based on Laplacian Score in Sparse Coding , 2011, MLDM.

[14]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[15]  Guillermo Sapiro,et al.  Dictionary learning and sparse coding for unsupervised clustering , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[16]  Guillermo Sapiro,et al.  Classification and clustering via dictionary learning with structured incoherence and shared features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Daniel W. C. Ho,et al.  Underdetermined blind source separation based on sparse representation , 2006, IEEE Transactions on Signal Processing.

[18]  Haibo He,et al.  Learning from Imbalanced Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[19]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[20]  David D. Lewis,et al.  Heterogeneous Uncertainty Sampling for Supervised Learning , 1994, ICML.

[21]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale $\ell_1$-Regularized Least Squares , 2007, IEEE Journal of Selected Topics in Signal Processing.

[22]  Guang Yang,et al.  Sparse-Representation-Based Classification with Structure-Preserving Dimension Reduction , 2014, Cognitive Computation.

[23]  Thomas Martinetz,et al.  Sparse Coding Neural Gas: Learning of overcomplete data representations , 2009, Neurocomputing.

[24]  Kjersti Engan,et al.  Multi-frame compression: theory and design , 2000, Signal Process..

[25]  Fei Wang,et al.  Compressed Nonnegative Sparse Coding , 2010, 2010 IEEE International Conference on Data Mining.

[26]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[27]  Wang-Q Lim,et al.  Sparse multidimensional representation using shearlets , 2005, SPIE Optics + Photonics.

[28]  Guillermo Sapiro,et al.  Online dictionary learning for sparse coding , 2009, ICML '09.

[29]  Tina Eliassi-Rad,et al.  Local Structural Features Threaten Privacy across Social Networks , 2013 .

[30]  Haibo He,et al.  Feature selection based on sparse imputation , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[31]  E. Candès,et al.  Curvelets: A Surprisingly Effective Nonadaptive Representation for Objects with Edges , 2000 .

[32]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[33]  Rebecca Hwa,et al.  Sample Selection for Statistical Parsing , 2004, CL.

[34]  Haibo He,et al.  DCPE co-training for classification , 2012, Neurocomputing.

[35]  Daniel S. Yeung,et al.  Localized generalization error based active learning for image annotation , 2008, 2008 IEEE International Conference on Systems, Man and Cybernetics.

[36]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[37]  Stefan Wrobel,et al.  Active Hidden Markov Models for Information Extraction , 2001, IDA.

[38]  H. Sebastian Seung,et al.  Query by committee , 1992, COLT '92.

[39]  Mark Craven,et al.  Active Learning with Real Annotation Costs , 2008 .

[40]  David Zhang,et al.  Fisher Discrimination Dictionary Learning for sparse representation , 2011, 2011 International Conference on Computer Vision.