Structured Sparse Priors for Image Classification

Model-based compressive sensing (CS) exploits the structure inherent in sparse signals for the design of better signal recovery algorithms. This information about structure is often captured in the form of a prior on the sparse coefficients, with the Laplacian being the most common such choice (leading to l1-norm minimization). Recent work has exploited the discriminative capability of sparse representations for image classification by employing class-specific dictionaries in the CS framework. Our contribution is a logical extension of these ideas into structured sparsity for classification. We introduce the notion of discriminative class-specific priors in conjunction with class specific dictionaries, specifically the spike-and-slab prior widely applied in Bayesian sparse regression. Significantly, the proposed framework takes the burden off the demand for abundant training image samples necessary for the success of sparsity-based classification schemes. We demonstrate this practical benefit of our approach in important applications, such as face recognition and object categorization.

[1]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[2]  E. George,et al.  Journal of the American Statistical Association is currently published by American Statistical Association. , 2007 .

[3]  Hugh Chipman,et al.  Bayesian variable selection with related predictors , 1995, bayes-an/9510001.

[4]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[5]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[6]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[7]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[8]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[9]  A. Martínez,et al.  The AR face databasae , 1998 .

[10]  J. Anthony Gualtieri,et al.  Support vector machines for hyperspectral remote sensing classification , 1999, Other Conferences.

[11]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[12]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[14]  Michael W. Marcellin,et al.  JPEG2000 - image compression fundamentals, standards and practice , 2002, The Kluwer International Series in Engineering and Computer Science.

[15]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[16]  David J. Kriegman,et al.  Clustering appearances of objects under varying illumination conditions , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[17]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[18]  J. S. Rao,et al.  Spike and slab variable selection: Frequentist and Bayesian strategies , 2005, math/0505633.

[19]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[20]  Richard G. Baraniuk,et al.  Distributed Compressed Sensing Dror , 2005 .

[21]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[22]  D. Donoho For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[23]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[24]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[25]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[26]  Jitendra Malik,et al.  SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[27]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[28]  Andrew Zisserman,et al.  A Visual Vocabulary for Flower Classification , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[29]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[30]  Lawrence Carin,et al.  Bayesian Compressive Sensing , 2008, IEEE Transactions on Signal Processing.

[31]  Junzhou Huang,et al.  Simultaneous image transformation and sparse representation recovery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  M. West,et al.  High-Dimensional Sparse Factor Modeling: Applications in Gene Expression Genomics , 2008, Journal of the American Statistical Association.

[33]  Yongsheng Gao,et al.  Face recognition across pose: A review , 2009, Pattern Recognit..

[34]  Lawrence Carin,et al.  Exploiting Structure in Wavelet-Based Bayesian Compressive Sensing , 2009, IEEE Transactions on Signal Processing.

[35]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[36]  Babak Hassibi,et al.  On the Reconstruction of Block-Sparse Signals With an Optimal Number of Measurements , 2008, IEEE Transactions on Signal Processing.

[37]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[38]  Mike E. Davies,et al.  Sampling Theorems for Signals From the Union of Finite-Dimensional Linear Subspaces , 2009, IEEE Transactions on Information Theory.

[39]  Volkan Cevher,et al.  Learning with Compressible Priors , 2009, NIPS.

[40]  Sebastian Nowozin,et al.  On feature combination for multiclass object classification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[41]  A. Madabhushi,et al.  Histopathological Image Analysis: A Review , 2009, IEEE Reviews in Biomedical Engineering.

[42]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Rabab Kreidieh Ward,et al.  Classification via group sparsity promoting regularization , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[44]  Garrison W. Cottrell,et al.  Robust classification of objects, faces, and flowers using natural image statistics , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[45]  Allen Y. Yang,et al.  Fast ℓ1-minimization algorithms and an application in robust face recognition: A review , 2010, 2010 IEEE International Conference on Image Processing.

[46]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[47]  Rabab Kreidieh Ward,et al.  Improved Group Sparse Classifier , 2010, Pattern Recognit. Lett..

[48]  Yihong Gong,et al.  Locality-constrained Linear Coding for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[49]  Volkan Cevher,et al.  Sparse Signal Recovery and Acquisition with Graphical Models , 2010, IEEE Signal Processing Magazine.

[50]  Shuicheng Yan,et al.  Visual classification with multi-task joint sparse representation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[51]  Aggelos K. Katsaggelos,et al.  Bayesian Compressive Sensing Using Laplace Priors , 2010, IEEE Transactions on Image Processing.

[52]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[53]  Yang Yu,et al.  Group sparsity based classification for cervigram segmentation , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[54]  Anil K. Jain,et al.  Handbook of Face Recognition, 2nd Edition , 2011 .

[55]  Hugues Talbot,et al.  A Memory Gradient algorithm for ℓ2 — ℓ0 regularization with applications to image restoration , 2011, 2011 18th IEEE International Conference on Image Processing.

[56]  Yonina C. Eldar,et al.  Structured Compressed Sensing: From Theory to Applications , 2011, IEEE Transactions on Signal Processing.

[57]  David Zhang,et al.  Fisher Discrimination Dictionary Learning for sparse representation , 2011, 2011 International Conference on Computer Vision.

[58]  Jian Yang,et al.  Robust sparse coding for face recognition , 2011, CVPR 2011.

[59]  T. Yen A majorization–minimization approach to variable selection using spike and slab priors , 2010, 1005.0891.

[60]  Rama Chellappa,et al.  Secure and Robust Iris Recognition Using Random Projections and Sparse Representations , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Miguel Lázaro-Gredilla,et al.  Spike and Slab Variational Inference for Multi-Task and Multiple Kernel Learning , 2011, NIPS.

[62]  Yonina C. Eldar,et al.  C-HiLasso: A Collaborative Hierarchical Sparse Modeling Framework , 2010, IEEE Transactions on Signal Processing.

[63]  Andrew Zisserman,et al.  BiCoS: A Bi-level co-segmentation method for image classification , 2011, 2011 International Conference on Computer Vision.

[64]  Yoshua Bengio,et al.  Large-Scale Feature Learning With Spike-and-Slab Sparse Coding , 2012, ICML.

[65]  Julien Mairal,et al.  Optimization with Sparsity-Inducing Penalties , 2011, Found. Trends Mach. Learn..

[66]  Hossein Mobahi,et al.  Toward a Practical Face Recognition System: Robust Alignment and Illumination by Sparse Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[67]  Katherine A. Heller,et al.  Evaluating Bayesian and L1 Approaches for Sparse Unsupervised Learning , 2011, ICML.

[68]  Thomas S. Huang,et al.  Joint-Structured-Sparsity-Based Classification for Multiple-Measurement Transient Acoustic Signals , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[69]  Shuicheng Yan,et al.  Visual Classification With Multitask Joint Sparse Representation , 2012, IEEE Transactions on Image Processing.

[70]  Hugues Talbot,et al.  A Majorize-Minimize Subspace Approach for ℓ2-ℓ0 Image Regularization , 2011, SIAM J. Imaging Sci..

[71]  Michael W. Marcellin,et al.  JPEG2000 - image compression fundamentals, standards and practice , 2013, The Kluwer international series in engineering and computer science.

[72]  Trac D. Tran,et al.  Structured sparse priors for image classification , 2013, ICIP.

[73]  Trac D. Tran,et al.  Hierarchical sparse modeling using Spike and Slab priors , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.