Learning Structured Twin-Incoherent Twin-Projective Latent Dictionary Pairs for Classification

In this paper, we extend the popular dictionary pair learning (DPL) into the scenario of twin-projective latent flexible DPL under a structured twin-incoherence. Technically, a novel framework called Twin-Projective Latent Flexible DPL (TP-DPL) is proposed, which minimizes the twin-incoherence constrained flexibly-relaxed reconstruction error to avoid the possible over-fitting issue and produce accurate reconstruction. In this setting, TP-DPL integrates the twin-incoherence based latent flexible DPL and the joint embedding of codes as well as salient features by twin-projection into a unified model in an adaptive neighborhood-preserving manner. Therefore, TP-DPL can unify the procedures of salient feature representation and classification. The twin-incoherence constraint on coefficients and features can explicitly ensure high intra-class compactness and inter-class separation over them. TP-DPL also integrates the adaptive weighting to preserve local neighborhood of both coefficients and salient features within each class explicitly. For efficiency, TP-DPL selects the Frobenius-norm and abandons the costly l0/l1-norm for group sparse representation. Another byproduct is that TP-DPL can directly apply the class-specific twin-projective reconstruction residual to compute the label of data. Extensive results on public databases show that TP-DPL can deliver the state-of-the-art performance.

[1]  Jiawei Han,et al.  Document clustering using locality preserving indexing , 2005, IEEE Transactions on Knowledge and Data Engineering.

[2]  Larry S. Davis,et al.  Submodular dictionary learning for sparse coding , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Kathrin Klamroth,et al.  Biconvex sets and optimization with biconvex functions: a survey and extensions , 2007, Math. Methods Oper. Res..

[4]  Shuicheng Yan,et al.  Jointly Learning Structured Analysis Discriminative Dictionary and Analysis Multiclass Classifier , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[7]  A. Martínez,et al.  The AR face databasae , 1998 .

[8]  Richard E. Wendell,et al.  Minimization of a Non-Separable Objective Function Subject to Disjoint Constraints , 1976, Oper. Res..

[9]  Lei Zhang,et al.  Projective dictionary pair learning for pattern classification , 2014, NIPS.

[10]  D. B. Gerham Characterizing virtual eigensignatures for general purpose face recognition , 1998 .

[11]  Baoxin Li,et al.  Discriminative K-SVD for dictionary learning in face recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Tommy W. S. Chow,et al.  Sparse Codes Auto-Extractor for Classification: A Joint Embedding and Dictionary Learning Framework for Representation , 2016, IEEE Transactions on Signal Processing.

[13]  Jan de Leeuw,et al.  Block-relaxation Algorithms in Statistics , 1994 .

[14]  Li Zhang,et al.  Structured Latent Label Consistent Dictionary Learning for Salient Machine Faults Representation-Based Robust Classification , 2017, IEEE Transactions on Industrial Informatics.

[15]  Jian Yang,et al.  A Locality-Constrained and Label Embedding Dictionary Learning Algorithm for Image Classification , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[16]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  D. B. Graham,et al.  Characterising Virtual Eigensignatures for General Purpose Face Recognition , 1998 .

[18]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[19]  Shutao Li,et al.  Contextual Online Dictionary Learning for Hyperspectral Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Vishal Monga,et al.  Fast Low-Rank Shared Dictionary Learning for Image Classification , 2016, IEEE Transactions on Image Processing.

[21]  Xuan Tang,et al.  Reconstructible Nonlinear Dimensionality Reduction via Joint Dictionary Learning , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Meng Wang,et al.  Robust Projective Low-Rank and Sparse Representation by Robust Dictionary Learning , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[23]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[24]  Rama Chellappa,et al.  Sparse dictionary-based representation and recognition of action attributes , 2011, 2011 International Conference on Computer Vision.

[25]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[26]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[27]  Lunke Fei,et al.  Robust Sparse Linear Discriminant Analysis , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[28]  Jan-Michael Frahm,et al.  Self-Expressive Dictionary Learning for Dynamic 3D Reconstruction , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[30]  David Zhang,et al.  Sparse Representation Based Fisher Discrimination Dictionary Learning for Image Classification , 2014, International Journal of Computer Vision.

[31]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[32]  Li Zhang,et al.  Joint Label Consistent Dictionary Learning and Adaptive Label Prediction for Semisupervised Machine Fault Classification , 2016, IEEE Transactions on Industrial Informatics.

[33]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[34]  Bernt Schiele,et al.  Analyzing appearance and contour based methods for object categorization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[35]  Guillermo Sapiro,et al.  Classification and clustering via dictionary learning with structured incoherence and shared features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[36]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[37]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[38]  Angshul Majumdar,et al.  Discriminative Robust Deep Dictionary Learning for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[40]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[41]  Ling Shao,et al.  Discriminative Fisher Embedding Dictionary Learning Algorithm for Object Recognition , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[42]  Donghui Wang,et al.  A Dictionary Learning Approach for Classification: Separating the Particularity and the Commonality , 2012, ECCV.

[43]  Volker Blanz,et al.  Component-Based Face Recognition with 3D Morphable Models , 2003, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[44]  Larry S. Davis,et al.  Label Consistent K-SVD: Learning a Discriminative Dictionary for Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Zhao Zhang,et al.  Joint Subspace Recovery and Enhanced Locality Driven Robust Flexible Discriminative Dictionary Learning , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[46]  Shuicheng Yan,et al.  Robust Locality-Constrained Label Consistent K-SVD by Joint Sparse Embedding , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[47]  Guangcan Liu,et al.  Scalable Block-Diagonal Locality-Constrained Projective Dictionary Learning , 2019, IJCAI.

[48]  Yan Zhang,et al.  Discriminative sparse flexible manifold embedding with novel graph for robust visual representation and label propagation , 2017, Pattern Recognit..