Synthesis linear classifier based analysis dictionary learning for pattern classification

Abstract Dictionary learning approaches have been widely applied to solve pattern classification problems and have achieved promising performance. However, most of works aim to learn a discriminative synthesis dictionary and sparse coding coefficients for classification. Until recent years, analysis dictionary learning began to attract interest from researchers. In this paper, we present a novel discriminative analysis dictionary learning frame, named Synthesis Linear Classifier based Analysis Dictionary Learning (SLC-ADL). Firstly, we incorporate a synthesis-linear-classifier-based error term into the basic analysis dictionary learning model, whose classification performance is obviously improved by making full use of the label information. Then, we develop an alternating iterative algorithm to solve the new model and obtain closed-form solutions leading to pretty competitive running efficiency. What is more, we design three classification schemes by fully exploiting the synthesis linear classifier. Finally, extensive comparison experiments on scene categorization, object classification, action recognition and face recognition clearly verify the classification performance of the proposed algorithm.

[1]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[2]  David G. Lowe,et al.  Local Naive Bayes Nearest Neighbor for image classification , 2011, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Wei Cao,et al.  Robust iris recognition using sparse error correction model and discriminative dictionary learning , 2014, Neurocomputing.

[4]  Ke Huang,et al.  Sparse Representation for Signal Classification , 2006, NIPS.

[5]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[6]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[7]  Li Shang,et al.  Immune K-SVD algorithm for dictionary learning in speech denoising , 2014, Neurocomputing.

[8]  Wei Dai,et al.  Analysis SimCO: A new algorithm for analysis dictionary learning , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[9]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[10]  Ming-Hsuan Yang,et al.  Top-down visual saliency via joint CRF and dictionary learning , 2012, CVPR.

[11]  Alain Rakotomamonjy,et al.  Applying alternating direction method of multipliers for constrained dictionary learning , 2013, Neurocomputing.

[12]  Shengping Zhang,et al.  Robust visual tracking based on online learning sparse representation , 2013, Neurocomputing.

[13]  Yoram Bresler,et al.  Structured Overcomplete Sparsifying Transform Learning with Convergence Guarantees and Applications , 2015, International Journal of Computer Vision.

[14]  Yoram Bresler,et al.  Efficient Blind Compressed Sensing Using Sparsifying Transforms with Convergence Guarantees and Application to Magnetic Resonance Imaging , 2015, SIAM J. Imaging Sci..

[15]  Qingming Huang,et al.  Multi-Level Discriminative Dictionary Learning With Application to Large Scale Image Classification , 2015, IEEE Transactions on Image Processing.

[16]  Shuyuan Yang,et al.  Multitask dictionary learning and sparse representation based single-image super-resolution reconstruction , 2011, Neurocomputing.

[17]  Richard E. Wendell,et al.  Minimization of a Non-Separable Objective Function Subject to Disjoint Constraints , 1976, Oper. Res..

[18]  Tieniu Tan,et al.  Robust Recovery of Corrupted Low-rank Matrix by Implicit Regularizers. , 2013, IEEE transactions on pattern analysis and machine intelligence.

[19]  Lei Zhang,et al.  Sparse representation or collaborative representation: Which helps face recognition? , 2011, 2011 International Conference on Computer Vision.

[20]  Yi Li,et al.  CRF with locality-consistent dictionary learning for semantic segmentation , 2015, 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR).

[21]  Tieniu Tan,et al.  Half-Quadratic-Based Iterative Minimization for Robust Sparse Representation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Guillermo Sapiro,et al.  Discriminative learned dictionaries for local image analysis , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Tieniu Tan,et al.  Robust Recovery of Corrupted Low-RankMatrix by Implicit Regularizers , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Lei Zhang,et al.  Metaface learning for sparse representation based face recognition , 2010, 2010 IEEE International Conference on Image Processing.

[25]  Mubarak Shah,et al.  Recognizing 50 human action categories of web videos , 2012, Machine Vision and Applications.

[26]  Robert Tibshirani,et al.  Discriminant Adaptive Nearest Neighbor Classification , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Baoxin Li,et al.  Discriminative K-SVD for dictionary learning in face recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[29]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[30]  Rama Chellappa,et al.  Analysis sparse coding models for image-based classification , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[31]  Donghui Wang,et al.  A Dictionary Learning Approach for Classification: Separating the Particularity and the Commonality , 2012, ECCV.

[32]  Michael Elad,et al.  Dictionary Learning for Analysis-Synthesis Thresholding , 2014, IEEE Transactions on Signal Processing.

[33]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Jason J. Corso,et al.  Action bank: A high-level representation of activity in video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Guillermo Sapiro,et al.  Classification and clustering via dictionary learning with structured incoherence and shared features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[36]  Yoram Bresler,et al.  Learning Sparsifying Transforms , 2013, IEEE Transactions on Signal Processing.

[37]  Jian Yang,et al.  Robust sparse coding for face recognition , 2011, CVPR 2011.

[38]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[39]  Xin Yao,et al.  Application of Genetic Algorithm and K-Nearest Neighbour Method in Real World Medical Fraud Detection Problem , 2000, J. Adv. Comput. Intell. Intell. Informatics.

[40]  Michael Elad,et al.  Analysis K-SVD: A Dictionary-Learning Algorithm for the Analysis Sparse Model , 2013, IEEE Transactions on Signal Processing.

[41]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, CVPR.

[42]  David Zhang,et al.  Fisher Discrimination Dictionary Learning for sparse representation , 2011, 2011 International Conference on Computer Vision.

[43]  Ender M. Eksioglu,et al.  K-SVD Meets Transform Learning: Transform K-SVD , 2014, IEEE Signal Processing Letters.

[44]  A. Martínez,et al.  The AR face databasae , 1998 .

[45]  Larry S. Davis,et al.  Label Consistent K-SVD: Learning a Discriminative Dictionary for Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Jianping Fan,et al.  Learning inter-related visual dictionary for object recognition , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Dimitrios Gunopulos,et al.  Large margin nearest neighbor classifiers , 2005, IEEE Transactions on Neural Networks.

[48]  Kathrin Klamroth,et al.  Biconvex sets and optimization with biconvex functions: a survey and extensions , 2007, Math. Methods Oper. Res..

[49]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[51]  Yunsong Guo,et al.  Metric Learning: A Support Vector Approach , 2008, ECML/PKDD.

[52]  Lei Zhang,et al.  Projective dictionary pair learning for pattern classification , 2014, NIPS.

[53]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.