Jointly discriminative projection and dictionary learning for domain adaptive collaborative representation-based classification

Abstract In recent years, collaborative representation-based classification (CRC) methods have shown impressive performance in many recognition tasks. However, when the training data have different distributions with the testing data, the performance of CRC will be degraded significantly. On the other hand, concatenating training data from different sources as a single data set will affect the performance of CRC, as the shift exists between the different source domains. To address these problems, in this paper, we propose a Jointly Discriminative projection and Dictionary learning for domain adaptive Collaborative Representation-based Classification method (JD2-CRC). As the distributions of different source domains may be dissimilar, the data from all domains are projected into a common feature subspace where the latent shared structures can be found. Then a compact dictionary is learned to represent the projected data well. To find the most suitable projection matrices and dictionary for CRC, we design the objective function of JD2-CRC,according to the classification rule of CRC in feature subspace, which minimizes the ratio of within-class reconstruction errors over between-class reconstruction errors. Different to traditional optimization methods, an effective optimization procedure is presented based on gradient descent. Thus, the obtained collaborative representations have a better discriminability and suit the classification rule of CRC well. The experimental results demonstrate that the proposed method can achieve superior performance against other state-of-the-art methods.

[1]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[2]  Yuan Shi,et al.  Geodesic flow kernel for unsupervised domain adaptation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[4]  Zhang Yi,et al.  Learning locality-constrained collaborative representation for robust face recognition , 2012, Pattern Recognit..

[5]  Korris Fu-Lai Chung,et al.  A trace ratio maximization approach to multiple kernel-based dimensionality reduction , 2014, Neural Networks.

[6]  Huan Wang,et al.  Dimension reduction using collaborative representation reconstruction based projections , 2016, Neurocomputing.

[7]  Jim Jing-Yan Wang,et al.  Supervised Transfer Sparse Coding , 2014, AAAI.

[8]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[9]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[10]  Zhenyu Wang,et al.  A collaborative representation based projections method for feature extraction , 2015, Pattern Recognit..

[11]  Mohammed Bellalij,et al.  The Trace Ratio Optimization Problem , 2012, SIAM Rev..

[12]  Huchuan Lu,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON IMAGE PROCESSING 1 Online Object Tracking with Sparse Prototypes , 2022 .

[13]  Hong Qiao,et al.  Efficient Fisher Discrimination Dictionary Learning , 2016, Signal Process..

[14]  Quansen Sun,et al.  Optimal Couple Projections for Domain Adaptive Sparse Representation-Based Classification , 2017, IEEE Transactions on Image Processing.

[15]  Kiran B. Raja,et al.  Improved ear verification after surgery - An approach based on collaborative representation of locally competitive features , 2018, Pattern Recognit..

[16]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[17]  A. Izenman Linear Discriminant Analysis , 2013 .

[18]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[19]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[21]  Rama Chellappa,et al.  Coupled Projections for Adaptation of Dictionaries , 2015, IEEE Transactions on Image Processing.

[22]  Jiawei Han,et al.  Orthogonal Laplacianfaces for Face Recognition , 2006, IEEE Transactions on Image Processing.

[23]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Jieping Ye,et al.  Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems , 2005, J. Mach. Learn. Res..

[25]  Tao Li,et al.  Collaborative representation based local discriminant projection for feature extraction , 2018, Digit. Signal Process..

[26]  Ying Shen,et al.  Towards contactless palmprint recognition: A novel device, a new benchmark, and a collaborative representation based identification approach , 2017, Pattern Recognit..

[27]  Wotao Yin,et al.  A feasible method for optimization with orthogonality constraints , 2013, Math. Program..

[28]  Sumit Chopra,et al.  DLID: Deep Learning for Domain Adaptation by Interpolating between Domains , 2013 .

[29]  Trevor Darrell,et al.  Adapting Visual Category Models to New Domains , 2010, ECCV.

[30]  G. Griffin,et al.  Caltech-256 Object Category Dataset , 2007 .

[31]  Rama Chellappa,et al.  Domain adaptation for object recognition: An unsupervised approach , 2011, 2011 International Conference on Computer Vision.

[32]  LinLin Shen,et al.  Joint and collaborative representation with local adaptive convolution feature for face recognition with single sample per person , 2017, Pattern Recognit..

[33]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[34]  Tinne Tuytelaars,et al.  Unsupervised Visual Domain Adaptation Using Subspace Alignment , 2013, 2013 IEEE International Conference on Computer Vision.

[35]  Ajmal S. Mian,et al.  Efficient classification with sparsity augmented collaborative representation , 2017, Pattern Recognit..

[36]  Melba M. Crawford,et al.  Manifold-Learning-Based Feature Extraction for Classification of Hyperspectral Data: A Review of Advances in Manifold Learning , 2014, IEEE Signal Processing Magazine.

[37]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[38]  Da-Zheng Feng,et al.  Enhanced regularized least square based discriminative projections for feature extraction , 2017, Signal Process..

[39]  Jian Yang,et al.  Sparse Representation Classifier Steered Discriminative Projection With Applications to Face Recognition , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[40]  Rama Chellappa,et al.  Domain adaptive sparse representation-based classification , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[41]  Lei Zhang,et al.  Sparse representation or collaborative representation: Which helps face recognition? , 2011, 2011 International Conference on Computer Vision.

[42]  Hongmei Chi,et al.  Competitive and collaborative representation for classification , 2020, Pattern Recognit. Lett..

[43]  Ivor W. Tsang,et al.  Learning With Augmented Features for Supervised and Semi-Supervised Heterogeneous Domain Adaptation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Rama Chellappa,et al.  Generalized Domain-Adaptive Dictionaries , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[47]  A. Martínez,et al.  The AR face databasae , 1998 .

[48]  Hossein Mobahi,et al.  Toward a Practical Face Recognition System: Robust Alignment and Illumination by Sparse Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[50]  Dong Xu,et al.  Trace Ratio vs. Ratio Trace for Dimensionality Reduction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Lai Wei,et al.  Kernel locality-constrained collaborative representation based discriminant analysis , 2014, Knowl. Based Syst..

[52]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[53]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Lai Wei,et al.  Optimized projection for Collaborative Representation based Classification and its applications to face recognition , 2016, Pattern Recognit. Lett..

[55]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.