SLR: Semi-Coupled Locality Constrained Representation for Very Low Resolution Face Recognition and Super Resolution

Although face recognition algorithms have been greatly successful recently, in real applications of very low-resolution (VLR) images, both super-resolution (SR) and recognition tasks are more challenging than those in high-resolution (HR) images. Given the rare discriminative information in VLR images, the one-to-many mapping relationship between HR and VLR images degrades the SR and recognition performances. In this paper, we propose a novel semi-coupled dictionary learning scheme to promote discriminative and representative abilities for face recognition and SR simultaneously by relaxing coupled dictionary learning. Specifically, we use semi-coupled locality-constrained representation to enhance the consistency between VLR and HR local manifold geometries, thereby overcoming the negative effects of one-to-many mapping. Given the learned task-oriented mapping function, we feed these discriminative features into a collaborative representation-based classifier to output their labels, and combine a locality-induced approach to hallucinate the HR images. Extensive experimental results demonstrate that the proposed approach outperforms a number of state-of-the-art face recognition and SR algorithms.

[1]  Zheng Wang,et al.  SRLSP: A Face Image Super-Resolution Algorithm Using Smooth Regression With Local Structure Prior , 2017, IEEE Transactions on Multimedia.

[2]  Luc Van Gool,et al.  Is image super-resolution helpful for other vision tasks? , 2015, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[3]  Ruimin Hu,et al.  CDMMA: Coupled discriminant multi-manifold analysis for matching low-resolution face images , 2016, Signal Process..

[4]  Zhongyuan Wang,et al.  Smart Monitoring Cameras Driven Intelligent Processing to Big Surveillance Video Data , 2018, IEEE Transactions on Big Data.

[5]  Maoguo Gong,et al.  Position-Patch Based Face Hallucination Using Convex Optimization , 2011, IEEE Signal Processing Letters.

[6]  Xiaogang Wang,et al.  Hallucinating face by eigentransformation , 2005, IEEE Trans. Syst. Man Cybern. Part C.

[7]  Ling Li,et al.  Face hallucination: How much it can improve face recognition , 2013, 2013 Australian Control Conference.

[8]  Hao Wang,et al.  Face hallucination using deep collaborative representation for local and non-local patches , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[9]  B.V.K. Vijaya Kumar,et al.  How Low Can You Go? Low Resolution Face Recognition Study Using Kernel Correlation Feature Analysis on the FRGCv2 dataset , 2006, 2006 Biometrics Symposium: Special Session on Research at the Biometric Consortium Conference.

[10]  Chun Qi,et al.  From Local Geometry to Global Structure: Learning Latent Subspace for Low-resolution Face Image Recognition , 2015, IEEE Signal Processing Letters.

[11]  Shiguang Shan,et al.  Low-Resolution Face Recognition via Coupled Locality Preserving Mappings , 2010, IEEE Signal Processing Letters.

[12]  Xiao-Yuan Jing,et al.  Super-resolution Person re-identification with semi-coupled low-rank discriminant dictionary learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Ruimin Hu,et al.  Facial Image Hallucination Through Coupled-Layer Neighbor Embedding , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Zixiang Xiong,et al.  Very Low-Resolution Face Recognition via Semi-Coupled Locality-Constrained Representation , 2016, 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS).

[15]  Yu-Chiang Frank Wang,et al.  Recognition at a long distance: Very low resolution face recognition and hallucination , 2015, 2015 International Conference on Biometrics (ICB).

[16]  Ruimin Hu,et al.  Face Hallucination Via Weighted Adaptive Sparse Regularization , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Yihong Gong,et al.  Locality-constrained Linear Coding for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Jean Ponce,et al.  Sparse Modeling for Image and Vision Processing , 2014, Found. Trends Comput. Graph. Vis..

[19]  Hong Chang,et al.  Super-resolution through neighbor embedding , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[20]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Zhuowen Tu,et al.  Robust Point Matching via Vector Field Consensus , 2014, IEEE Transactions on Image Processing.

[22]  Lei Zhang,et al.  Sparse representation or collaborative representation: Which helps face recognition? , 2011, 2011 International Conference on Computer Vision.

[23]  Zixiang Xiong,et al.  Robust and efficient face recognition via low-rank supported extreme learning machine , 2018, Multimedia Tools and Applications.

[24]  Jiwen Lu,et al.  PCANet: A Simple Deep Learning Baseline for Image Classification? , 2014, IEEE Transactions on Image Processing.

[25]  Zixiang Xiong,et al.  DLML: Deep linear mappings learning for face super-resolution with nonlocal-patch , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[26]  Pong C. Yuen,et al.  Very low resolution face recognition problem , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[27]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Ruimin Hu,et al.  Noise Robust Face Hallucination via Locality-Constrained Representation , 2014, IEEE Transactions on Multimedia.

[29]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Chun Qi,et al.  Global consistency, local sparsity and pixel correlation: A unified framework for face hallucination , 2014, Pattern Recognit..

[31]  Q. M. Jonathan Wu,et al.  Low-resolution face recognition: a review , 2013, The Visual Computer.

[32]  Pablo H. Hennings-Yeomans,et al.  Simultaneous super-resolution and feature extraction for recognition of low-resolution faces , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Jie Li,et al.  Image super-resolution: The techniques, applications, and future , 2016, Signal Process..

[35]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Thomas S. Huang,et al.  Studying Very Low Resolution Recognition Using Deep Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Takeo Kanade,et al.  Hallucinating faces , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[38]  Hua Huang,et al.  Super-Resolution Method for Face Recognition Using Nonlinear Mappings on Coherent Features , 2011, IEEE Transactions on Neural Networks.

[39]  Zhang Yi,et al.  Learning locality-constrained collaborative representation for robust face recognition , 2012, Pattern Recognit..

[40]  Baowen Xu,et al.  Super-resolution Person re-identification with semi-coupled low-rank discriminant dictionary learning , 2015, CVPR.

[41]  Hao Wang,et al.  Face hallucination using region-based deep convolutional networks , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[42]  Junjun Jiang,et al.  Noise Robust Face Image Super-Resolution Through Smooth Sparse Representation , 2017, IEEE Transactions on Cybernetics.

[43]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[44]  Quan Pan,et al.  Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Patrick J. Flynn,et al.  Pose-robust recognition of low-resolution face images , 2013, CVPR 2011.

[46]  Jun Yu,et al.  Click Prediction for Web Image Reranking Using Multimodal Sparse Coding , 2014, IEEE Transactions on Image Processing.

[47]  Nicu Sebe,et al.  Neighborhood issue in single-frame image super-resolution , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[48]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[49]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[50]  Patrick J. Flynn,et al.  Multidimensional Scaling for Matching Low-Resolution Face Images , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Shengcai Liao,et al.  Nighttime Face Recognition at Long Distance: Cross-Distance and Cross-Spectral Matching , 2012, ACCV.

[52]  Zhenghong Yu,et al.  Fast non-rigid image feature matching for agricultural UAV via probabilistic inference with regularization techniques , 2017, Comput. Electron. Agric..

[53]  Ruimin Hu,et al.  Position-Patch Based Face Hallucination via Locality-Constrained Representation , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[54]  Dong Yue,et al.  Locality-Constrained Double Low-Rank Representation for Effective Face Hallucination , 2016, IEEE Access.

[55]  Guoying Zhao,et al.  Hallucinating Face Image by Regularization Models in High-Resolution Feature Space , 2018, IEEE Transactions on Image Processing.

[56]  Chun Qi,et al.  Hallucinating face by position-patch , 2010, Pattern Recognit..

[57]  Junjun Jiang,et al.  Locality Preserving Matching , 2017, IJCAI.

[58]  Yihong Gong,et al.  Resolution-Invariant Image Representation for Content-Based Zooming , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[59]  Chun Qi,et al.  Kernel-Based Face Hallucination via Dual Regularization Priors , 2015, IEEE Signal Processing Letters.

[60]  Kin-Man Lam,et al.  Simultaneous Hallucination and Recognition of Low-Resolution Faces Based on Singular Value Decomposition , 2015, IEEE Transactions on Circuits and Systems for Video Technology.